Machine Learning Engineer Nanodegree¶
Capstone¶
Project- Corn Commodity Futures Price Predictor¶
In this project we will try to predict closing weekly price of Corn Commodity Futures. In order to perform this prediction we will create a dataset that includes weekly Corn Futures closing prices as well as Long Open Interest and Short Open Interest of Processors/Users( sometimes they are called Commercials) from COT reports and by using this dataset we will try to predict next week’s prices.
1. Data Sets¶
Historical Futures Prices: Corn Futures, Continuous Contract #1. Non-adjusted price based on spot-month continuous contract calculations. Raw data from CME:
Can be found here
Commitment of Traders - CORN (CBT) - Futures Only (002602)
Can be found here
Data has been downloaded and stored in \Data folder:
- .\data\CHRIS-CME_C1.csv - Corn Futures Prices data
- .\data\CFTC-002602_F_ALL.csv - Commitment of Traders data
In [1]:
import warnings
warnings.filterwarnings('ignore')
In [2]:
import pandas as pd
import numpy as np
from IPython.core.display import display, HTML
pd.options.display.max_colwidth = 500 # You need this, otherwise pandas
# will limit your HTML strings to 50 characters
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)
pd.options.mode.chained_assignment = None # default='warn'
from matplotlib import pyplot
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from math import sqrt
from numpy import concatenate
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
import cufflinks as cf
import plotly.tools as tls
init_notebook_mode(connected=True)
cf.go_offline()
Using TensorFlow backend. C:\Users\zilvi\Anaconda3\envs\zil_tensorflow\lib\site-packages\plotly\graph_objs\_deprecations.py:558: DeprecationWarning: plotly.graph_objs.YAxis is deprecated. Please replace it with one of the following more specific types - plotly.graph_objs.layout.YAxis - plotly.graph_objs.layout.scene.YAxis C:\Users\zilvi\Anaconda3\envs\zil_tensorflow\lib\site-packages\plotly\graph_objs\_deprecations.py:531: DeprecationWarning: plotly.graph_objs.XAxis is deprecated. Please replace it with one of the following more specific types - plotly.graph_objs.layout.XAxis - plotly.graph_objs.layout.scene.XAxis
2. Prepare and Explore Data¶
In [3]:
df_fut_orig = pd.read_csv('data\CHRIS-CME_C1.csv')
df_fut_orig.head(n=5)
Out[3]:
| Date | Open | High | Low | Last | Change | Settle | Volume | Previous_Day_Open_Interest | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2018-07-10 | 344.25 | 344.75 | 336.25 | 339.50 | 6.00 | 339.75 | 2668.0 | 2186.0 |
| 1 | 2018-07-09 | 346.00 | 348.50 | 342.50 | 346.00 | 6.00 | 345.75 | 3190.0 | 2969.0 |
| 2 | 2018-07-06 | 342.00 | 352.25 | 342.00 | 350.75 | 8.25 | 351.75 | 3068.0 | 3959.0 |
| 3 | 2018-07-05 | 345.50 | 348.75 | 341.50 | 342.50 | 0.75 | 343.50 | 3302.0 | 4812.0 |
| 4 | 2018-07-03 | 340.25 | 345.25 | 339.25 | 343.25 | 5.25 | 342.75 | 3048.0 | 5687.0 |
In [4]:
# Display a description of the dataset
display(df_fut_orig.describe())
| Open | High | Low | Last | Change | Settle | Volume | Previous_Day_Open_Interest | |
|---|---|---|---|---|---|---|---|---|
| count | 3033.000000 | 3034.000000 | 3034.000000 | 3034.000000 | 1081.000000 | 3034.000000 | 3034.000000 | 3034.00000 |
| mean | 457.095038 | 462.322924 | 451.795485 | 456.920040 | 3.950324 | 456.979318 | 103905.200396 | 352140.90145 |
| std | 140.338892 | 142.056030 | 138.436196 | 140.243019 | 3.415126 | 140.204571 | 73993.219920 | 248565.85531 |
| min | 219.000000 | 220.750000 | 216.750000 | 219.000000 | 0.000000 | 219.000000 | 0.000000 | 107.00000 |
| 25% | 360.000000 | 363.000000 | 356.250000 | 359.500000 | 1.500000 | 359.750000 | 40172.750000 | 107559.25000 |
| 50% | 388.500000 | 392.000000 | 383.500000 | 388.750000 | 3.000000 | 389.000000 | 102567.000000 | 365073.00000 |
| 75% | 565.500000 | 573.562500 | 557.375000 | 564.625000 | 5.500000 | 564.625000 | 152391.250000 | 556408.50000 |
| max | 830.250000 | 843.750000 | 822.750000 | 831.250000 | 30.750000 | 831.250000 | 538170.000000 | 858696.00000 |
In [5]:
df_fut_orig['Date'] = pd.to_datetime(df_fut_orig['Date'])
df_fut_orig.set_index('Date',inplace=True)
df_fut_orig = df_fut_orig.sort_values('Date')
Plot Corn Futures Price Series using Plotly
In [6]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_original_price_series(df_fut_orig)
Seems there are some rows where Volume=0, lets find out more about these rows
In [7]:
df_fut_orig[df_fut_orig['Volume']<1]
Out[7]:
| Open | High | Low | Last | Change | Settle | Volume | Previous_Day_Open_Interest | |
|---|---|---|---|---|---|---|---|---|
| Date | ||||||||
| 2007-04-05 | 359.75 | 367.50 | 357.25 | 366.00 | NaN | 366.00 | 0.0 | 354349.0 |
| 2012-04-06 | 658.25 | 658.25 | 658.25 | 658.25 | NaN | 658.25 | 0.0 | 401521.0 |
| 2015-04-03 | 386.50 | 386.50 | 386.50 | 386.50 | NaN | 386.50 | 0.0 | 470964.0 |
Since we will resample daily prices into weekly prices , lets drop those rows.
In [8]:
# drop outliers
df_fut_orig.drop(df_fut_orig[df_fut_orig.Volume<1].index, inplace=True)
In [9]:
df_cot_orig = pd.read_csv('data\CFTC-002602_F_ALL.csv')
display(df_cot_orig.head())
| Date | Open_Interest | Producer_Merchant_Processor_User_Longs | Producer_Merchant_Processor_User_Shorts | Swap Dealer Longs | Swap Dealer Shorts | Swap Dealer Spreads | Money Manager Longs | Money Manager Shorts | Money Manager Spreads | Other Reportable Longs | Other Reportable Shorts | Other Reportable Spreads | Total Reportable Longs | Total Reportable Shorts | Non Reportable Longs | Non Reportable Shorts | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2018-07-10 | 1818055.0 | 500172.0 | 750062.0 | 208128.0 | 39513.0 | 99477.0 | 263353.0 | 404297.0 | 154286.0 | 320946.0 | 70682.0 | 98709.0 | 1645071.0 | 1617026.0 | 172984.0 | 201029.0 |
| 1 | 2018-07-03 | 1830330.0 | 484257.0 | 773851.0 | 210341.0 | 36927.0 | 100340.0 | 274795.0 | 382191.0 | 149756.0 | 322256.0 | 66508.0 | 119627.0 | 1661372.0 | 1629200.0 | 168958.0 | 201130.0 |
| 2 | 2018-06-26 | 1885804.0 | 513100.0 | 840177.0 | 223131.0 | 32763.0 | 91972.0 | 287061.0 | 377825.0 | 153461.0 | 330396.0 | 58283.0 | 116745.0 | 1715866.0 | 1671226.0 | 169938.0 | 214578.0 |
| 3 | 2018-06-19 | 1992169.0 | 525197.0 | 920764.0 | 222105.0 | 41144.0 | 99285.0 | 299377.0 | 356828.0 | 163454.0 | 379025.0 | 56652.0 | 135078.0 | 1823521.0 | 1773205.0 | 168648.0 | 218964.0 |
| 4 | 2018-06-12 | 1963233.0 | 488666.0 | 917204.0 | 235249.0 | 37674.0 | 93281.0 | 292054.0 | 304292.0 | 172623.0 | 363918.0 | 65030.0 | 147098.0 | 1792889.0 | 1737202.0 | 170344.0 | 226031.0 |
In [10]:
display(df_cot_orig.describe())
| Open_Interest | Producer_Merchant_Processor_User_Longs | Producer_Merchant_Processor_User_Shorts | Swap Dealer Longs | Swap Dealer Shorts | Swap Dealer Spreads | Money Manager Longs | Money Manager Shorts | Money Manager Spreads | Other Reportable Longs | Other Reportable Shorts | Other Reportable Spreads | Total Reportable Longs | Total Reportable Shorts | Non Reportable Longs | Non Reportable Shorts | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 6.310000e+02 | 631.000000 | 6.310000e+02 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 631.000000 | 6.310000e+02 | 6.310000e+02 | 631.000000 | 631.000000 |
| mean | 1.292201e+06 | 270795.049128 | 6.268425e+05 | 290792.497623 | 20337.034865 | 33260.068146 | 236884.269414 | 137472.426307 | 94546.356577 | 140931.890650 | 70914.334390 | 85505.109350 | 1.152715e+06 | 1.068878e+06 | 139485.541997 | 223322.976228 |
| std | 2.095471e+05 | 68976.221600 | 1.554272e+05 | 53203.484072 | 18944.008732 | 22912.567257 | 67454.195123 | 109465.025186 | 32739.133163 | 51939.690903 | 26360.863384 | 29682.425476 | 1.939790e+05 | 2.060080e+05 | 23718.957966 | 29824.710288 |
| min | 7.482520e+05 | 102373.000000 | 2.972960e+05 | 186981.000000 | 0.000000 | 4397.000000 | 96989.000000 | 6714.000000 | 29130.000000 | 49809.000000 | 25905.000000 | 27592.000000 | 6.379810e+05 | 5.689510e+05 | 78578.000000 | 156086.000000 |
| 25% | 1.192226e+06 | 226595.000000 | 5.235930e+05 | 255196.500000 | 6524.000000 | 13978.000000 | 186366.500000 | 47947.000000 | 72018.500000 | 104764.000000 | 53331.000000 | 62690.000000 | 1.055362e+06 | 9.573815e+05 | 121829.500000 | 198860.500000 |
| 50% | 1.301506e+06 | 262823.000000 | 6.112810e+05 | 276337.000000 | 15239.000000 | 27209.000000 | 225682.000000 | 95548.000000 | 91850.000000 | 140343.000000 | 66261.000000 | 82705.000000 | 1.166372e+06 | 1.067548e+06 | 136966.000000 | 227337.000000 |
| 75% | 1.398275e+06 | 314224.000000 | 7.058555e+05 | 321265.500000 | 28178.000000 | 48009.500000 | 287331.000000 | 211154.000000 | 113803.000000 | 175846.000000 | 83448.500000 | 106077.500000 | 1.247976e+06 | 1.180280e+06 | 153542.500000 | 246903.000000 |
| max | 1.992169e+06 | 525197.000000 | 1.001517e+06 | 422803.000000 | 95591.000000 | 113775.000000 | 431569.000000 | 447470.000000 | 231064.000000 | 379025.000000 | 173322.000000 | 181385.000000 | 1.825238e+06 | 1.773205e+06 | 206821.000000 | 293948.000000 |
Drop unnecessary columns columns and resample data¶
In [11]:
df_fut=df_fut_orig.drop(columns=[clmn for i,clmn in enumerate(df_fut_orig.columns) if i not in [5,6,7] ],axis=1)
display(df_fut.head())
| Settle | Volume | Previous_Day_Open_Interest | |
|---|---|---|---|
| Date | |||
| 2006-06-16 | 235.50 | 56486.0 | 203491.0 |
| 2006-06-19 | 229.75 | 51299.0 | 190044.0 |
| 2006-06-20 | 229.75 | 41605.0 | 175859.0 |
| 2006-06-21 | 232.75 | 29803.0 | 162348.0 |
| 2006-06-22 | 230.50 | 28687.0 | 147658.0 |
In [12]:
s_settle =df_fut['Settle'].resample('W').last()
s_volume =df_fut['Volume'].resample('W').last()
df_fut_weekly = pd.concat([s_settle,s_volume], axis=1)
display(df_fut_weekly.head())
| Settle | Volume | |
|---|---|---|
| Date | ||
| 2006-06-18 | 235.50 | 56486.0 |
| 2006-06-25 | 228.25 | 28361.0 |
| 2006-07-02 | 235.50 | 30519.0 |
| 2006-07-09 | 241.00 | 13057.0 |
| 2006-07-16 | 253.50 | 2460.0 |
In [13]:
df_cot=df_cot_orig.drop(columns=[clmn for i,clmn in enumerate(df_cot_orig.columns) if i not in [0,1,2,3 ]],axis=1)
df_cot.rename(index=str, columns={"Producer_Merchant_Processor_User_Longs": "Longs", \
"Producer_Merchant_Processor_User_Shorts": "Shorts"},inplace=True)
df_cot['Date'] = pd.to_datetime(df_cot['Date'])
df_cot.set_index('Date',inplace=True)
display(df_cot.head())
| Open_Interest | Longs | Shorts | |
|---|---|---|---|
| Date | |||
| 2018-07-10 | 1818055.0 | 500172.0 | 750062.0 |
| 2018-07-03 | 1830330.0 | 484257.0 | 773851.0 |
| 2018-06-26 | 1885804.0 | 513100.0 | 840177.0 |
| 2018-06-19 | 1992169.0 | 525197.0 | 920764.0 |
| 2018-06-12 | 1963233.0 | 488666.0 | 917204.0 |
In [14]:
s_longs =df_cot['Longs'].resample('W').last()
s_shorts =df_cot['Shorts'].resample('W').last()
s_open_interest =df_cot['Open_Interest'].resample('W').last()
df_cot_weekly = pd.concat([s_open_interest,s_longs, s_shorts], axis=1)
display(df_cot_weekly.head(5))
| Open_Interest | Longs | Shorts | |
|---|---|---|---|
| Date | |||
| 2006-06-18 | 1320155.0 | 209662.0 | 699163.0 |
| 2006-06-25 | 1321520.0 | 224476.0 | 666688.0 |
| 2006-07-02 | 1329400.0 | 234769.0 | 645735.0 |
| 2006-07-09 | 1327482.0 | 220552.0 | 648405.0 |
| 2006-07-16 | 1333225.0 | 216968.0 | 673110.0 |
In [15]:
df_weekly = pd.merge(df_fut_weekly,df_cot_weekly, on='Date')
display(df_weekly.head(5))
| Settle | Volume | Open_Interest | Longs | Shorts | |
|---|---|---|---|---|---|
| Date | |||||
| 2006-06-18 | 235.50 | 56486.0 | 1320155.0 | 209662.0 | 699163.0 |
| 2006-06-25 | 228.25 | 28361.0 | 1321520.0 | 224476.0 | 666688.0 |
| 2006-07-02 | 235.50 | 30519.0 | 1329400.0 | 234769.0 | 645735.0 |
| 2006-07-09 | 241.00 | 13057.0 | 1327482.0 | 220552.0 | 648405.0 |
| 2006-07-16 | 253.50 | 2460.0 | 1333225.0 | 216968.0 | 673110.0 |
In [16]:
# Display a description of the dataset
display(df_weekly.describe())
| Settle | Volume | Open_Interest | Longs | Shorts | |
|---|---|---|---|---|---|
| count | 631.000000 | 631.000000 | 6.310000e+02 | 631.000000 | 6.310000e+02 |
| mean | 456.978605 | 100835.204437 | 1.292201e+06 | 270795.049128 | 6.268425e+05 |
| std | 140.242112 | 72466.341538 | 2.095471e+05 | 68976.221600 | 1.554272e+05 |
| min | 219.750000 | 132.000000 | 7.482520e+05 | 102373.000000 | 2.972960e+05 |
| 25% | 359.500000 | 34822.500000 | 1.192226e+06 | 226595.000000 | 5.235930e+05 |
| 50% | 389.250000 | 101209.000000 | 1.301506e+06 | 262823.000000 | 6.112810e+05 |
| 75% | 560.375000 | 150341.000000 | 1.398275e+06 | 314224.000000 | 7.058555e+05 |
| max | 824.500000 | 369522.000000 | 1.992169e+06 | 525197.000000 | 1.001517e+06 |
In [17]:
# rest index since we need row numbers for splitting
df_weekly_idx_date=df_weekly.copy()
df_weekly.reset_index(inplace=True)
3. Visualise Data¶
In [18]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_weekly_combined_series_by_date(df_weekly)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [19]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_weekly_combined_series_by_trading_week(df_weekly)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [20]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_grouped_by_year_data(df_weekly_idx_date,"Stacked Plots of Price by Year")
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [21]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.lag_plot(df_weekly,"Lag Plot")
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
4. Normalise the data using minmaxscaler¶
In [22]:
scaler = MinMaxScaler(feature_range=(0, 1))
values = df_weekly.loc[:, df_weekly.columns != 'Date'].values
scaled = scaler.fit_transform(values)
5. Split data into training, validation and test sets¶
In [23]:
validation_start=df_weekly[df_weekly['Date'] >= pd.to_datetime('2017-01-01')].index[0]
testing_start=df_weekly[df_weekly['Date'] >= pd.to_datetime('2018-01-01')].index[0]
In [24]:
print("validation start",validation_start)
print("testing start",testing_start)
validation start 550 testing start 603
In [25]:
# print data to double check
#print(df_weekly.iloc[validation_start])
#print(df_weekly.iloc[testing_start])
In [26]:
%load_ext autoreload
%autoreload 2
import data_preparer
reframed = data_preparer.series_to_supervised(scaled, 1, 1)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [27]:
# drop columns we don't want to predict
reframed.drop(reframed.columns[[6,7,8,9]], axis=1, inplace=True)
In [28]:
display(reframed.head())
| var1(t-1) | var2(t-1) | var3(t-1) | var4(t-1) | var5(t-1) | var1(t) | |
|---|---|---|---|---|---|---|
| 1 | 0.026044 | 0.152560 | 0.459760 | 0.253744 | 0.570655 | 0.014055 |
| 2 | 0.014055 | 0.076421 | 0.460857 | 0.288780 | 0.524540 | 0.026044 |
| 3 | 0.026044 | 0.082263 | 0.467192 | 0.313123 | 0.494786 | 0.035138 |
| 4 | 0.035138 | 0.034990 | 0.465650 | 0.279499 | 0.498578 | 0.055808 |
| 5 | 0.055808 | 0.006302 | 0.470267 | 0.271023 | 0.533659 | 0.028938 |
6. Define and Fit Model¶
In [66]:
%load_ext autoreload
%autoreload 2
import data_preparer
train_X, train_y, validation_X, validation_y,test_X, test_y = data_preparer.split_data(reframed,validation_start,testing_start)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [67]:
%load_ext autoreload
%autoreload 2
import models
model,history=models.basic_lstm_model(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload Train on 550 samples, validate on 53 samples Epoch 1/500 - 14s - loss: 0.5592 - val_loss: 0.3919 Epoch 2/500 - 0s - loss: 0.5173 - val_loss: 0.3437 Epoch 3/500 - 0s - loss: 0.4764 - val_loss: 0.2974 Epoch 4/500 - 0s - loss: 0.4374 - val_loss: 0.2533 Epoch 5/500 - 0s - loss: 0.4009 - val_loss: 0.2120 Epoch 6/500 - 0s - loss: 0.3678 - val_loss: 0.1729 Epoch 7/500 - 0s - loss: 0.3363 - val_loss: 0.1351 Epoch 8/500 - 0s - loss: 0.3058 - val_loss: 0.0985 Epoch 9/500 - 0s - loss: 0.2770 - val_loss: 0.0638 Epoch 10/500 - 0s - loss: 0.2515 - val_loss: 0.0362 Epoch 11/500 - 0s - loss: 0.2331 - val_loss: 0.0267 Epoch 12/500 - 0s - loss: 0.2226 - val_loss: 0.0308 Epoch 13/500 - 0s - loss: 0.2165 - val_loss: 0.0382 Epoch 14/500 - 0s - loss: 0.2126 - val_loss: 0.0453 Epoch 15/500 - 0s - loss: 0.2100 - val_loss: 0.0516 Epoch 16/500 - 0s - loss: 0.2083 - val_loss: 0.0560 Epoch 17/500 - 0s - loss: 0.2071 - val_loss: 0.0595 Epoch 18/500 - 0s - loss: 0.2062 - val_loss: 0.0622 Epoch 19/500 - 0s - loss: 0.2054 - val_loss: 0.0641 Epoch 20/500 - 0s - loss: 0.2047 - val_loss: 0.0655 Epoch 21/500 - 0s - loss: 0.2041 - val_loss: 0.0665 Epoch 22/500 - 0s - loss: 0.2036 - val_loss: 0.0669 Epoch 23/500 - 0s - loss: 0.2031 - val_loss: 0.0670 Epoch 24/500 - 0s - loss: 0.2026 - val_loss: 0.0672 Epoch 25/500 - 0s - loss: 0.2021 - val_loss: 0.0674 Epoch 26/500 - 0s - loss: 0.2016 - val_loss: 0.0675 Epoch 27/500 - 0s - loss: 0.2012 - val_loss: 0.0676 Epoch 28/500 - 0s - loss: 0.2007 - val_loss: 0.0677 Epoch 29/500 - 0s - loss: 0.2002 - val_loss: 0.0678 Epoch 30/500 - 0s - loss: 0.1998 - val_loss: 0.0677 Epoch 31/500 - 0s - loss: 0.1994 - val_loss: 0.0675 Epoch 32/500 - 0s - loss: 0.1989 - val_loss: 0.0672 Epoch 33/500 - 0s - loss: 0.1985 - val_loss: 0.0669 Epoch 34/500 - 0s - loss: 0.1980 - val_loss: 0.0665 Epoch 35/500 - 0s - loss: 0.1976 - val_loss: 0.0662 Epoch 36/500 - 0s - loss: 0.1972 - val_loss: 0.0659 Epoch 37/500 - 0s - loss: 0.1967 - val_loss: 0.0655 Epoch 38/500 - 0s - loss: 0.1963 - val_loss: 0.0652 Epoch 39/500 - 0s - loss: 0.1959 - val_loss: 0.0648 Epoch 40/500 - 0s - loss: 0.1954 - val_loss: 0.0644 Epoch 41/500 - 0s - loss: 0.1950 - val_loss: 0.0639 Epoch 42/500 - 0s - loss: 0.1945 - val_loss: 0.0634 Epoch 43/500 - 0s - loss: 0.1941 - val_loss: 0.0630 Epoch 44/500 - 0s - loss: 0.1936 - val_loss: 0.0626 Epoch 45/500 - 0s - loss: 0.1931 - val_loss: 0.0621 Epoch 46/500 - 0s - loss: 0.1927 - val_loss: 0.0616 Epoch 47/500 - 0s - loss: 0.1922 - val_loss: 0.0612 Epoch 48/500 - 0s - loss: 0.1917 - val_loss: 0.0608 Epoch 49/500 - 0s - loss: 0.1912 - val_loss: 0.0604 Epoch 50/500 - 0s - loss: 0.1907 - val_loss: 0.0601 Epoch 51/500 - 0s - loss: 0.1902 - val_loss: 0.0598 Epoch 52/500 - 0s - loss: 0.1896 - val_loss: 0.0595 Epoch 53/500 - 0s - loss: 0.1891 - val_loss: 0.0592 Epoch 54/500 - 0s - loss: 0.1885 - val_loss: 0.0589 Epoch 55/500 - 0s - loss: 0.1879 - val_loss: 0.0587 Epoch 56/500 - 0s - loss: 0.1873 - val_loss: 0.0585 Epoch 57/500 - 0s - loss: 0.1867 - val_loss: 0.0583 Epoch 58/500 - 0s - loss: 0.1860 - val_loss: 0.0582 Epoch 59/500 - 0s - loss: 0.1854 - val_loss: 0.0582 Epoch 60/500 - 0s - loss: 0.1847 - val_loss: 0.0582 Epoch 61/500 - 0s - loss: 0.1840 - val_loss: 0.0582 Epoch 62/500 - 0s - loss: 0.1832 - val_loss: 0.0580 Epoch 63/500 - 0s - loss: 0.1825 - val_loss: 0.0578 Epoch 64/500 - 0s - loss: 0.1817 - val_loss: 0.0577 Epoch 65/500 - 0s - loss: 0.1808 - val_loss: 0.0577 Epoch 66/500 - 0s - loss: 0.1800 - val_loss: 0.0576 Epoch 67/500 - 0s - loss: 0.1791 - val_loss: 0.0574 Epoch 68/500 - 0s - loss: 0.1781 - val_loss: 0.0572 Epoch 69/500 - 0s - loss: 0.1772 - val_loss: 0.0570 Epoch 70/500 - 0s - loss: 0.1762 - val_loss: 0.0565 Epoch 71/500 - 0s - loss: 0.1752 - val_loss: 0.0560 Epoch 72/500 - 0s - loss: 0.1741 - val_loss: 0.0552 Epoch 73/500 - 0s - loss: 0.1730 - val_loss: 0.0546 Epoch 74/500 - 0s - loss: 0.1719 - val_loss: 0.0541 Epoch 75/500 - 0s - loss: 0.1707 - val_loss: 0.0534 Epoch 76/500 - 0s - loss: 0.1695 - val_loss: 0.0527 Epoch 77/500 - 0s - loss: 0.1683 - val_loss: 0.0522 Epoch 78/500 - 0s - loss: 0.1670 - val_loss: 0.0518 Epoch 79/500 - 0s - loss: 0.1656 - val_loss: 0.0515 Epoch 80/500 - 0s - loss: 0.1642 - val_loss: 0.0510 Epoch 81/500 - 0s - loss: 0.1628 - val_loss: 0.0505 Epoch 82/500 - 0s - loss: 0.1613 - val_loss: 0.0499 Epoch 83/500 - 0s - loss: 0.1597 - val_loss: 0.0495 Epoch 84/500 - 0s - loss: 0.1581 - val_loss: 0.0491 Epoch 85/500 - 0s - loss: 0.1564 - val_loss: 0.0489 Epoch 86/500 - 0s - loss: 0.1546 - val_loss: 0.0483 Epoch 87/500 - 0s - loss: 0.1528 - val_loss: 0.0477 Epoch 88/500 - 0s - loss: 0.1509 - val_loss: 0.0471 Epoch 89/500 - 0s - loss: 0.1489 - val_loss: 0.0469 Epoch 90/500 - 0s - loss: 0.1469 - val_loss: 0.0470 Epoch 91/500 - 0s - loss: 0.1447 - val_loss: 0.0471 Epoch 92/500 - 0s - loss: 0.1425 - val_loss: 0.0472 Epoch 93/500 - 0s - loss: 0.1402 - val_loss: 0.0470 Epoch 94/500 - 0s - loss: 0.1378 - val_loss: 0.0468 Epoch 95/500 - 0s - loss: 0.1353 - val_loss: 0.0466 Epoch 96/500 - 0s - loss: 0.1328 - val_loss: 0.0460 Epoch 97/500 - 0s - loss: 0.1303 - val_loss: 0.0456 Epoch 98/500 - 0s - loss: 0.1277 - val_loss: 0.0452 Epoch 99/500 - 0s - loss: 0.1250 - val_loss: 0.0446 Epoch 100/500 - 0s - loss: 0.1222 - val_loss: 0.0442 Epoch 101/500 - 0s - loss: 0.1193 - val_loss: 0.0440 Epoch 102/500 - 0s - loss: 0.1164 - val_loss: 0.0437 Epoch 103/500 - 0s - loss: 0.1134 - val_loss: 0.0435 Epoch 104/500 - 0s - loss: 0.1103 - val_loss: 0.0435 Epoch 105/500 - 0s - loss: 0.1071 - val_loss: 0.0438 Epoch 106/500 - 0s - loss: 0.1039 - val_loss: 0.0443 Epoch 107/500 - 0s - loss: 0.1007 - val_loss: 0.0448 Epoch 108/500 - 0s - loss: 0.0976 - val_loss: 0.0453 Epoch 109/500 - 0s - loss: 0.0945 - val_loss: 0.0459 Epoch 110/500 - 0s - loss: 0.0915 - val_loss: 0.0467 Epoch 111/500 - 0s - loss: 0.0886 - val_loss: 0.0477 Epoch 112/500 - 0s - loss: 0.0856 - val_loss: 0.0486 Epoch 113/500 - 0s - loss: 0.0827 - val_loss: 0.0500 Epoch 114/500 - 0s - loss: 0.0804 - val_loss: 0.0517 Epoch 115/500 - 0s - loss: 0.0783 - val_loss: 0.0534 Epoch 116/500 - 0s - loss: 0.0761 - val_loss: 0.0539 Epoch 117/500 - 0s - loss: 0.0747 - val_loss: 0.0551 Epoch 118/500 - 0s - loss: 0.0732 - val_loss: 0.0562 Epoch 119/500 - 0s - loss: 0.0717 - val_loss: 0.0570 Epoch 120/500 - 0s - loss: 0.0705 - val_loss: 0.0583 Epoch 121/500 - 0s - loss: 0.0693 - val_loss: 0.0590 Epoch 122/500 - 0s - loss: 0.0679 - val_loss: 0.0589 Epoch 123/500 - 0s - loss: 0.0670 - val_loss: 0.0597 Epoch 124/500 - 0s - loss: 0.0660 - val_loss: 0.0602 Epoch 125/500 - 0s - loss: 0.0649 - val_loss: 0.0599 Epoch 126/500 - 0s - loss: 0.0640 - val_loss: 0.0601 Epoch 127/500 - 0s - loss: 0.0630 - val_loss: 0.0599 Epoch 128/500 - 0s - loss: 0.0621 - val_loss: 0.0597 Epoch 129/500 - 0s - loss: 0.0614 - val_loss: 0.0594 Epoch 130/500 - 0s - loss: 0.0606 - val_loss: 0.0593 Epoch 131/500 - 0s - loss: 0.0598 - val_loss: 0.0592 Epoch 132/500 - 0s - loss: 0.0590 - val_loss: 0.0587 Epoch 133/500 - 0s - loss: 0.0583 - val_loss: 0.0588 Epoch 134/500 - 0s - loss: 0.0576 - val_loss: 0.0586 Epoch 135/500 - 0s - loss: 0.0568 - val_loss: 0.0582 Epoch 136/500 - 0s - loss: 0.0561 - val_loss: 0.0579 Epoch 137/500 - 0s - loss: 0.0554 - val_loss: 0.0573 Epoch 138/500 - 0s - loss: 0.0548 - val_loss: 0.0567 Epoch 139/500 - 0s - loss: 0.0542 - val_loss: 0.0563 Epoch 140/500 - 0s - loss: 0.0536 - val_loss: 0.0562 Epoch 141/500 - 0s - loss: 0.0529 - val_loss: 0.0552 Epoch 142/500 - 0s - loss: 0.0524 - val_loss: 0.0547 Epoch 143/500 - 0s - loss: 0.0518 - val_loss: 0.0544 Epoch 144/500 - 0s - loss: 0.0512 - val_loss: 0.0539 Epoch 145/500 - 0s - loss: 0.0507 - val_loss: 0.0535 Epoch 146/500 - 0s - loss: 0.0501 - val_loss: 0.0527 Epoch 147/500 - 0s - loss: 0.0496 - val_loss: 0.0520 Epoch 148/500 - 0s - loss: 0.0491 - val_loss: 0.0516 Epoch 149/500 - 0s - loss: 0.0485 - val_loss: 0.0511 Epoch 150/500 - 0s - loss: 0.0480 - val_loss: 0.0505 Epoch 151/500 - 0s - loss: 0.0476 - val_loss: 0.0502 Epoch 152/500 - 0s - loss: 0.0471 - val_loss: 0.0496 Epoch 153/500 - 0s - loss: 0.0466 - val_loss: 0.0488 Epoch 154/500 - 0s - loss: 0.0461 - val_loss: 0.0483 Epoch 155/500 - 0s - loss: 0.0457 - val_loss: 0.0474 Epoch 156/500 - 0s - loss: 0.0453 - val_loss: 0.0467 Epoch 157/500 - 0s - loss: 0.0449 - val_loss: 0.0461 Epoch 158/500 - 0s - loss: 0.0445 - val_loss: 0.0451 Epoch 159/500 - 0s - loss: 0.0441 - val_loss: 0.0445 Epoch 160/500 - 0s - loss: 0.0437 - val_loss: 0.0438 Epoch 161/500 - 0s - loss: 0.0433 - val_loss: 0.0429 Epoch 162/500 - 0s - loss: 0.0430 - val_loss: 0.0423 Epoch 163/500 - 0s - loss: 0.0426 - val_loss: 0.0415 Epoch 164/500 - 0s - loss: 0.0422 - val_loss: 0.0411 Epoch 165/500 - 0s - loss: 0.0419 - val_loss: 0.0401 Epoch 166/500 - 0s - loss: 0.0416 - val_loss: 0.0394 Epoch 167/500 - 0s - loss: 0.0413 - val_loss: 0.0388 Epoch 168/500 - 0s - loss: 0.0410 - val_loss: 0.0381 Epoch 169/500 - 0s - loss: 0.0407 - val_loss: 0.0377 Epoch 170/500 - 0s - loss: 0.0404 - val_loss: 0.0374 Epoch 171/500 - 0s - loss: 0.0401 - val_loss: 0.0365 Epoch 172/500 - 0s - loss: 0.0399 - val_loss: 0.0361 Epoch 173/500 - 0s - loss: 0.0396 - val_loss: 0.0359 Epoch 174/500 - 0s - loss: 0.0394 - val_loss: 0.0352 Epoch 175/500 - 0s - loss: 0.0392 - val_loss: 0.0348 Epoch 176/500 - 0s - loss: 0.0389 - val_loss: 0.0344 Epoch 177/500 - 0s - loss: 0.0387 - val_loss: 0.0336 Epoch 178/500 - 0s - loss: 0.0385 - val_loss: 0.0331 Epoch 179/500 - 0s - loss: 0.0383 - val_loss: 0.0324 Epoch 180/500 - 0s - loss: 0.0381 - val_loss: 0.0322 Epoch 181/500 - 0s - loss: 0.0379 - val_loss: 0.0309 Epoch 182/500 - 0s - loss: 0.0377 - val_loss: 0.0312 Epoch 183/500 - 0s - loss: 0.0375 - val_loss: 0.0302 Epoch 184/500 - 0s - loss: 0.0373 - val_loss: 0.0306 Epoch 185/500 - 0s - loss: 0.0371 - val_loss: 0.0298 Epoch 186/500 - 0s - loss: 0.0370 - val_loss: 0.0297 Epoch 187/500 - 0s - loss: 0.0369 - val_loss: 0.0295 Epoch 188/500 - 0s - loss: 0.0367 - val_loss: 0.0291 Epoch 189/500 - 0s - loss: 0.0366 - val_loss: 0.0285 Epoch 190/500 - 0s - loss: 0.0364 - val_loss: 0.0282 Epoch 191/500 - 0s - loss: 0.0364 - val_loss: 0.0280 Epoch 192/500 - 0s - loss: 0.0362 - val_loss: 0.0279 Epoch 193/500 - 0s - loss: 0.0361 - val_loss: 0.0267 Epoch 194/500 - 0s - loss: 0.0360 - val_loss: 0.0271 Epoch 195/500 - 0s - loss: 0.0359 - val_loss: 0.0263 Epoch 196/500 - 0s - loss: 0.0357 - val_loss: 0.0260 Epoch 197/500 - 0s - loss: 0.0356 - val_loss: 0.0259 Epoch 198/500 - 0s - loss: 0.0355 - val_loss: 0.0255 Epoch 199/500 - 0s - loss: 0.0354 - val_loss: 0.0252 Epoch 200/500 - 0s - loss: 0.0353 - val_loss: 0.0247 Epoch 201/500 - 0s - loss: 0.0352 - val_loss: 0.0244 Epoch 202/500 - 0s - loss: 0.0351 - val_loss: 0.0242 Epoch 203/500 - 0s - loss: 0.0350 - val_loss: 0.0237 Epoch 204/500 - 0s - loss: 0.0349 - val_loss: 0.0235 Epoch 205/500 - 0s - loss: 0.0348 - val_loss: 0.0233 Epoch 206/500 - 0s - loss: 0.0346 - val_loss: 0.0227 Epoch 207/500 - 0s - loss: 0.0345 - val_loss: 0.0223 Epoch 208/500 - 0s - loss: 0.0344 - val_loss: 0.0221 Epoch 209/500 - 0s - loss: 0.0343 - val_loss: 0.0219 Epoch 210/500 - 0s - loss: 0.0343 - val_loss: 0.0217 Epoch 211/500 - 0s - loss: 0.0342 - val_loss: 0.0214 Epoch 212/500 - 0s - loss: 0.0341 - val_loss: 0.0213 Epoch 213/500 - 0s - loss: 0.0340 - val_loss: 0.0209 Epoch 214/500 - 0s - loss: 0.0339 - val_loss: 0.0207 Epoch 215/500 - 0s - loss: 0.0338 - val_loss: 0.0207 Epoch 216/500 - 0s - loss: 0.0337 - val_loss: 0.0204 Epoch 217/500 - 0s - loss: 0.0337 - val_loss: 0.0202 Epoch 218/500 - 0s - loss: 0.0336 - val_loss: 0.0201 Epoch 219/500 - 0s - loss: 0.0335 - val_loss: 0.0199 Epoch 220/500 - 0s - loss: 0.0334 - val_loss: 0.0196 Epoch 221/500 - 0s - loss: 0.0334 - val_loss: 0.0195 Epoch 222/500 - 0s - loss: 0.0333 - val_loss: 0.0194 Epoch 223/500 - 0s - loss: 0.0332 - val_loss: 0.0193 Epoch 224/500 - 0s - loss: 0.0332 - val_loss: 0.0194 Epoch 225/500 - 0s - loss: 0.0331 - val_loss: 0.0192 Epoch 226/500 - 0s - loss: 0.0330 - val_loss: 0.0191 Epoch 227/500 - 0s - loss: 0.0329 - val_loss: 0.0190 Epoch 228/500 - 0s - loss: 0.0329 - val_loss: 0.0189 Epoch 229/500 - 0s - loss: 0.0328 - val_loss: 0.0189 Epoch 230/500 - 0s - loss: 0.0327 - val_loss: 0.0187 Epoch 231/500 - 0s - loss: 0.0327 - val_loss: 0.0188 Epoch 232/500 - 0s - loss: 0.0326 - val_loss: 0.0185 Epoch 233/500 - 0s - loss: 0.0326 - val_loss: 0.0187 Epoch 234/500 - 0s - loss: 0.0325 - val_loss: 0.0183 Epoch 235/500 - 0s - loss: 0.0325 - val_loss: 0.0184 Epoch 236/500 - 0s - loss: 0.0325 - val_loss: 0.0183 Epoch 237/500 - 0s - loss: 0.0324 - val_loss: 0.0182 Epoch 238/500 - 0s - loss: 0.0324 - val_loss: 0.0180 Epoch 239/500 - 0s - loss: 0.0323 - val_loss: 0.0184 Epoch 240/500 - 0s - loss: 0.0323 - val_loss: 0.0178 Epoch 241/500 - 0s - loss: 0.0322 - val_loss: 0.0181 Epoch 242/500 - 0s - loss: 0.0321 - val_loss: 0.0177 Epoch 243/500 - 0s - loss: 0.0321 - val_loss: 0.0178 Epoch 244/500 - 0s - loss: 0.0321 - val_loss: 0.0178 Epoch 245/500 - 0s - loss: 0.0320 - val_loss: 0.0178 Epoch 246/500 - 0s - loss: 0.0320 - val_loss: 0.0174 Epoch 247/500 - 0s - loss: 0.0319 - val_loss: 0.0179 Epoch 248/500 - 0s - loss: 0.0319 - val_loss: 0.0173 Epoch 249/500 - 0s - loss: 0.0318 - val_loss: 0.0174 Epoch 250/500 - 0s - loss: 0.0318 - val_loss: 0.0172 Epoch 251/500 - 0s - loss: 0.0317 - val_loss: 0.0174 Epoch 252/500 - 0s - loss: 0.0317 - val_loss: 0.0172 Epoch 253/500 - 0s - loss: 0.0316 - val_loss: 0.0170 Epoch 254/500 - 0s - loss: 0.0316 - val_loss: 0.0170 Epoch 255/500 - 0s - loss: 0.0316 - val_loss: 0.0171 Epoch 256/500 - 0s - loss: 0.0314 - val_loss: 0.0168 Epoch 257/500 - 0s - loss: 0.0314 - val_loss: 0.0168 Epoch 258/500 - 0s - loss: 0.0314 - val_loss: 0.0168 Epoch 259/500 - 0s - loss: 0.0313 - val_loss: 0.0166 Epoch 260/500 - 0s - loss: 0.0313 - val_loss: 0.0166 Epoch 261/500 - 0s - loss: 0.0312 - val_loss: 0.0166 Epoch 262/500 - 0s - loss: 0.0312 - val_loss: 0.0165 Epoch 263/500 - 0s - loss: 0.0312 - val_loss: 0.0166 Epoch 264/500 - 0s - loss: 0.0311 - val_loss: 0.0164 Epoch 265/500 - 0s - loss: 0.0310 - val_loss: 0.0162 Epoch 266/500 - 0s - loss: 0.0310 - val_loss: 0.0167 Epoch 267/500 - 0s - loss: 0.0310 - val_loss: 0.0162 Epoch 268/500 - 0s - loss: 0.0310 - val_loss: 0.0163 Epoch 269/500 - 0s - loss: 0.0309 - val_loss: 0.0159 Epoch 270/500 - 0s - loss: 0.0309 - val_loss: 0.0161 Epoch 271/500 - 0s - loss: 0.0309 - val_loss: 0.0161 Epoch 272/500 - 0s - loss: 0.0308 - val_loss: 0.0160 Epoch 273/500 - 0s - loss: 0.0307 - val_loss: 0.0156 Epoch 274/500 - 0s - loss: 0.0307 - val_loss: 0.0164 Epoch 275/500 - 0s - loss: 0.0307 - val_loss: 0.0159 Epoch 276/500 - 0s - loss: 0.0306 - val_loss: 0.0157 Epoch 277/500 - 0s - loss: 0.0306 - val_loss: 0.0161 Epoch 278/500 - 0s - loss: 0.0306 - val_loss: 0.0160 Epoch 279/500 - 0s - loss: 0.0305 - val_loss: 0.0159 Epoch 280/500 - 0s - loss: 0.0304 - val_loss: 0.0156 Epoch 281/500 - 0s - loss: 0.0304 - val_loss: 0.0160 Epoch 282/500 - 0s - loss: 0.0304 - val_loss: 0.0158 Epoch 283/500 - 0s - loss: 0.0303 - val_loss: 0.0157 Epoch 284/500 - 0s - loss: 0.0303 - val_loss: 0.0158 Epoch 285/500 - 0s - loss: 0.0303 - val_loss: 0.0158 Epoch 286/500 - 0s - loss: 0.0303 - val_loss: 0.0156 Epoch 287/500 - 0s - loss: 0.0302 - val_loss: 0.0157 Epoch 288/500 - 0s - loss: 0.0302 - val_loss: 0.0156 Epoch 289/500 - 0s - loss: 0.0301 - val_loss: 0.0155 Epoch 290/500 - 0s - loss: 0.0302 - val_loss: 0.0156 Epoch 291/500 - 0s - loss: 0.0300 - val_loss: 0.0154 Epoch 292/500 - 0s - loss: 0.0301 - val_loss: 0.0157 Epoch 293/500 - 0s - loss: 0.0300 - val_loss: 0.0154 Epoch 294/500 - 0s - loss: 0.0300 - val_loss: 0.0155 Epoch 295/500 - 0s - loss: 0.0299 - val_loss: 0.0153 Epoch 296/500 - 0s - loss: 0.0299 - val_loss: 0.0154 Epoch 297/500 - 0s - loss: 0.0298 - val_loss: 0.0152 Epoch 298/500 - 0s - loss: 0.0299 - val_loss: 0.0153 Epoch 299/500 - 0s - loss: 0.0298 - val_loss: 0.0152 Epoch 300/500 - 0s - loss: 0.0298 - val_loss: 0.0153 Epoch 301/500 - 0s - loss: 0.0297 - val_loss: 0.0153 Epoch 302/500 - 0s - loss: 0.0297 - val_loss: 0.0152 Epoch 303/500 - 0s - loss: 0.0297 - val_loss: 0.0151 Epoch 304/500 - 0s - loss: 0.0296 - val_loss: 0.0151 Epoch 305/500 - 0s - loss: 0.0296 - val_loss: 0.0152 Epoch 306/500 - 0s - loss: 0.0296 - val_loss: 0.0151 Epoch 307/500 - 0s - loss: 0.0295 - val_loss: 0.0151 Epoch 308/500 - 0s - loss: 0.0295 - val_loss: 0.0150 Epoch 309/500 - 0s - loss: 0.0295 - val_loss: 0.0151 Epoch 310/500 - 0s - loss: 0.0294 - val_loss: 0.0150 Epoch 311/500 - 0s - loss: 0.0294 - val_loss: 0.0151 Epoch 312/500 - 0s - loss: 0.0293 - val_loss: 0.0149 Epoch 313/500 - 0s - loss: 0.0293 - val_loss: 0.0151 Epoch 314/500 - 0s - loss: 0.0293 - val_loss: 0.0150 Epoch 315/500 - 0s - loss: 0.0293 - val_loss: 0.0150 Epoch 316/500 - 0s - loss: 0.0292 - val_loss: 0.0150 Epoch 317/500 - 0s - loss: 0.0292 - val_loss: 0.0150 Epoch 318/500 - 0s - loss: 0.0291 - val_loss: 0.0150 Epoch 319/500 - 0s - loss: 0.0292 - val_loss: 0.0149 Epoch 320/500 - 0s - loss: 0.0291 - val_loss: 0.0149 Epoch 321/500 - 0s - loss: 0.0290 - val_loss: 0.0150 Epoch 322/500 - 0s - loss: 0.0291 - val_loss: 0.0149 Epoch 323/500 - 0s - loss: 0.0290 - val_loss: 0.0148 Epoch 324/500 - 0s - loss: 0.0290 - val_loss: 0.0149 Epoch 325/500 - 0s - loss: 0.0290 - val_loss: 0.0148 Epoch 326/500 - 0s - loss: 0.0289 - val_loss: 0.0148 Epoch 327/500 - 0s - loss: 0.0289 - val_loss: 0.0148 Epoch 328/500 - 0s - loss: 0.0289 - val_loss: 0.0149 Epoch 329/500 - 0s - loss: 0.0288 - val_loss: 0.0148 Epoch 330/500 - 0s - loss: 0.0288 - val_loss: 0.0148 Epoch 331/500 - 0s - loss: 0.0288 - val_loss: 0.0148 Epoch 332/500 - 0s - loss: 0.0288 - val_loss: 0.0150 Epoch 333/500 - 0s - loss: 0.0287 - val_loss: 0.0148 Epoch 334/500 - 0s - loss: 0.0287 - val_loss: 0.0148 Epoch 335/500 - 0s - loss: 0.0287 - val_loss: 0.0148 Epoch 336/500 - 0s - loss: 0.0286 - val_loss: 0.0149 Epoch 337/500 - 0s - loss: 0.0286 - val_loss: 0.0147 Epoch 338/500 - 0s - loss: 0.0286 - val_loss: 0.0148 Epoch 339/500 - 0s - loss: 0.0286 - val_loss: 0.0147 Epoch 340/500 - 0s - loss: 0.0286 - val_loss: 0.0147 Epoch 341/500 - 0s - loss: 0.0285 - val_loss: 0.0147 Epoch 342/500 - 0s - loss: 0.0285 - val_loss: 0.0146 Epoch 343/500 - 0s - loss: 0.0285 - val_loss: 0.0146 Epoch 344/500 - 0s - loss: 0.0285 - val_loss: 0.0145 Epoch 345/500 - 0s - loss: 0.0285 - val_loss: 0.0146 Epoch 346/500 - 0s - loss: 0.0284 - val_loss: 0.0146 Epoch 347/500 - 0s - loss: 0.0284 - val_loss: 0.0146 Epoch 348/500 - 0s - loss: 0.0284 - val_loss: 0.0145 Epoch 349/500 - 0s - loss: 0.0284 - val_loss: 0.0146 Epoch 350/500 - 0s - loss: 0.0283 - val_loss: 0.0146 Epoch 351/500 - 0s - loss: 0.0283 - val_loss: 0.0145 Epoch 352/500 - 0s - loss: 0.0283 - val_loss: 0.0144 Epoch 353/500 - 0s - loss: 0.0283 - val_loss: 0.0144 Epoch 354/500 - 0s - loss: 0.0283 - val_loss: 0.0145 Epoch 355/500 - 0s - loss: 0.0283 - val_loss: 0.0145 Epoch 356/500 - 0s - loss: 0.0282 - val_loss: 0.0144 Epoch 357/500 - 0s - loss: 0.0282 - val_loss: 0.0144 Epoch 358/500 - 0s - loss: 0.0282 - val_loss: 0.0143 Epoch 359/500 - 0s - loss: 0.0282 - val_loss: 0.0144 Epoch 360/500 - 0s - loss: 0.0282 - val_loss: 0.0143 Epoch 361/500 - 0s - loss: 0.0282 - val_loss: 0.0144 Epoch 362/500 - 0s - loss: 0.0281 - val_loss: 0.0143 Epoch 363/500 - 0s - loss: 0.0281 - val_loss: 0.0143 Epoch 364/500 - 0s - loss: 0.0281 - val_loss: 0.0143 Epoch 365/500 - 0s - loss: 0.0281 - val_loss: 0.0142 Epoch 366/500 - 0s - loss: 0.0281 - val_loss: 0.0142 Epoch 367/500 - 0s - loss: 0.0281 - val_loss: 0.0142 Epoch 368/500 - 0s - loss: 0.0280 - val_loss: 0.0141 Epoch 369/500 - 0s - loss: 0.0281 - val_loss: 0.0144 Epoch 370/500 - 0s - loss: 0.0280 - val_loss: 0.0141 Epoch 371/500 - 0s - loss: 0.0280 - val_loss: 0.0141 Epoch 372/500 - 0s - loss: 0.0280 - val_loss: 0.0141 Epoch 373/500 - 0s - loss: 0.0279 - val_loss: 0.0140 Epoch 374/500 - 0s - loss: 0.0279 - val_loss: 0.0141 Epoch 375/500 - 0s - loss: 0.0279 - val_loss: 0.0141 Epoch 376/500 - 0s - loss: 0.0279 - val_loss: 0.0140 Epoch 377/500 - 0s - loss: 0.0279 - val_loss: 0.0142 Epoch 378/500 - 0s - loss: 0.0279 - val_loss: 0.0143 Epoch 379/500 - 0s - loss: 0.0278 - val_loss: 0.0139 Epoch 380/500 - 0s - loss: 0.0279 - val_loss: 0.0143 Epoch 381/500 - 0s - loss: 0.0278 - val_loss: 0.0141 Epoch 382/500 - 0s - loss: 0.0278 - val_loss: 0.0139 Epoch 383/500 - 0s - loss: 0.0278 - val_loss: 0.0142 Epoch 384/500 - 0s - loss: 0.0278 - val_loss: 0.0141 Epoch 385/500 - 0s - loss: 0.0277 - val_loss: 0.0140 Epoch 386/500 - 0s - loss: 0.0277 - val_loss: 0.0139 Epoch 387/500 - 0s - loss: 0.0277 - val_loss: 0.0143 Epoch 388/500 - 0s - loss: 0.0277 - val_loss: 0.0142 Epoch 389/500 - 0s - loss: 0.0277 - val_loss: 0.0139 Epoch 390/500 - 0s - loss: 0.0277 - val_loss: 0.0139 Epoch 391/500 - 0s - loss: 0.0277 - val_loss: 0.0141 Epoch 392/500 - 0s - loss: 0.0277 - val_loss: 0.0142 Epoch 393/500 - 0s - loss: 0.0276 - val_loss: 0.0139 Epoch 394/500 - 0s - loss: 0.0276 - val_loss: 0.0140 Epoch 395/500 - 0s - loss: 0.0276 - val_loss: 0.0142 Epoch 396/500 - 0s - loss: 0.0276 - val_loss: 0.0143 Epoch 397/500 - 0s - loss: 0.0276 - val_loss: 0.0140 Epoch 398/500 - 0s - loss: 0.0276 - val_loss: 0.0140 Epoch 399/500 - 0s - loss: 0.0276 - val_loss: 0.0139 Epoch 400/500 - 0s - loss: 0.0275 - val_loss: 0.0140 Epoch 401/500 - 0s - loss: 0.0275 - val_loss: 0.0140 Epoch 402/500 - 0s - loss: 0.0275 - val_loss: 0.0139 Epoch 403/500 - 0s - loss: 0.0275 - val_loss: 0.0140 Epoch 404/500 - 0s - loss: 0.0275 - val_loss: 0.0138 Epoch 405/500 - 0s - loss: 0.0275 - val_loss: 0.0141 Epoch 406/500 - 0s - loss: 0.0275 - val_loss: 0.0141 Epoch 407/500 - 0s - loss: 0.0274 - val_loss: 0.0139 Epoch 408/500 - 0s - loss: 0.0274 - val_loss: 0.0137 Epoch 409/500 - 0s - loss: 0.0275 - val_loss: 0.0144 Epoch 410/500 - 0s - loss: 0.0274 - val_loss: 0.0139 Epoch 411/500 - 0s - loss: 0.0274 - val_loss: 0.0139 Epoch 412/500 - 0s - loss: 0.0274 - val_loss: 0.0138 Epoch 413/500 - 0s - loss: 0.0274 - val_loss: 0.0140 Epoch 414/500 - 0s - loss: 0.0274 - val_loss: 0.0138 Epoch 415/500 - 0s - loss: 0.0274 - val_loss: 0.0138 Epoch 416/500 - 0s - loss: 0.0274 - val_loss: 0.0138 Epoch 417/500 - 0s - loss: 0.0274 - val_loss: 0.0140 Epoch 418/500 - 0s - loss: 0.0273 - val_loss: 0.0138 Epoch 419/500 - 0s - loss: 0.0273 - val_loss: 0.0138 Epoch 420/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 421/500 - 0s - loss: 0.0274 - val_loss: 0.0139 Epoch 422/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 423/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 424/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 425/500 - 0s - loss: 0.0273 - val_loss: 0.0138 Epoch 426/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 427/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 428/500 - 0s - loss: 0.0273 - val_loss: 0.0139 Epoch 429/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 430/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 431/500 - 0s - loss: 0.0272 - val_loss: 0.0136 Epoch 432/500 - 0s - loss: 0.0272 - val_loss: 0.0136 Epoch 433/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 434/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 435/500 - 0s - loss: 0.0272 - val_loss: 0.0135 Epoch 436/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 437/500 - 0s - loss: 0.0272 - val_loss: 0.0136 Epoch 438/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 439/500 - 0s - loss: 0.0272 - val_loss: 0.0137 Epoch 440/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 441/500 - 0s - loss: 0.0271 - val_loss: 0.0136 Epoch 442/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 443/500 - 0s - loss: 0.0271 - val_loss: 0.0136 Epoch 444/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 445/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 446/500 - 0s - loss: 0.0271 - val_loss: 0.0135 Epoch 447/500 - 0s - loss: 0.0271 - val_loss: 0.0136 Epoch 448/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 449/500 - 0s - loss: 0.0271 - val_loss: 0.0135 Epoch 450/500 - 0s - loss: 0.0271 - val_loss: 0.0136 Epoch 451/500 - 0s - loss: 0.0271 - val_loss: 0.0135 Epoch 452/500 - 0s - loss: 0.0271 - val_loss: 0.0134 Epoch 453/500 - 0s - loss: 0.0271 - val_loss: 0.0134 Epoch 454/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 455/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 456/500 - 0s - loss: 0.0271 - val_loss: 0.0136 Epoch 457/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 458/500 - 0s - loss: 0.0270 - val_loss: 0.0133 Epoch 459/500 - 0s - loss: 0.0270 - val_loss: 0.0135 Epoch 460/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 461/500 - 0s - loss: 0.0270 - val_loss: 0.0133 Epoch 462/500 - 0s - loss: 0.0270 - val_loss: 0.0133 Epoch 463/500 - 0s - loss: 0.0270 - val_loss: 0.0133 Epoch 464/500 - 0s - loss: 0.0270 - val_loss: 0.0135 Epoch 465/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 466/500 - 0s - loss: 0.0269 - val_loss: 0.0133 Epoch 467/500 - 0s - loss: 0.0270 - val_loss: 0.0132 Epoch 468/500 - 0s - loss: 0.0270 - val_loss: 0.0134 Epoch 469/500 - 0s - loss: 0.0269 - val_loss: 0.0132 Epoch 470/500 - 0s - loss: 0.0269 - val_loss: 0.0133 Epoch 471/500 - 0s - loss: 0.0269 - val_loss: 0.0132 Epoch 472/500 - 0s - loss: 0.0269 - val_loss: 0.0133 Epoch 473/500 - 0s - loss: 0.0269 - val_loss: 0.0132 Epoch 474/500 - 0s - loss: 0.0269 - val_loss: 0.0132 Epoch 475/500 - 0s - loss: 0.0269 - val_loss: 0.0132 Epoch 476/500 - 0s - loss: 0.0269 - val_loss: 0.0131 Epoch 477/500 - 0s - loss: 0.0269 - val_loss: 0.0130 Epoch 478/500 - 0s - loss: 0.0269 - val_loss: 0.0131 Epoch 479/500 - 0s - loss: 0.0269 - val_loss: 0.0131 Epoch 480/500 - 0s - loss: 0.0269 - val_loss: 0.0131 Epoch 481/500 - 0s - loss: 0.0268 - val_loss: 0.0132 Epoch 482/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 483/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 484/500 - 0s - loss: 0.0268 - val_loss: 0.0130 Epoch 485/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 486/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 487/500 - 0s - loss: 0.0268 - val_loss: 0.0129 Epoch 488/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 489/500 - 0s - loss: 0.0268 - val_loss: 0.0129 Epoch 490/500 - 0s - loss: 0.0268 - val_loss: 0.0129 Epoch 491/500 - 0s - loss: 0.0268 - val_loss: 0.0130 Epoch 492/500 - 0s - loss: 0.0268 - val_loss: 0.0130 Epoch 493/500 - 0s - loss: 0.0267 - val_loss: 0.0129 Epoch 494/500 - 0s - loss: 0.0267 - val_loss: 0.0129 Epoch 495/500 - 0s - loss: 0.0268 - val_loss: 0.0129 Epoch 496/500 - 0s - loss: 0.0268 - val_loss: 0.0129 Epoch 497/500 - 0s - loss: 0.0267 - val_loss: 0.0129 Epoch 498/500 - 0s - loss: 0.0267 - val_loss: 0.0129 Epoch 499/500 - 0s - loss: 0.0267 - val_loss: 0.0128 Epoch 500/500 - 0s - loss: 0.0267 - val_loss: 0.0130
In [68]:
pyplot.plot(history['loss'], label='train')
pyplot.plot(history['val_loss'], label='validation')
pyplot.legend()
pyplot.show()
In [69]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
inv_yhat, inv_y, rmse=models.make_lstm_prediction(validation_X,validation_y,model,scaler)
print('LSTM Model on Validation Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload LSTM Model on Validation Data RMSE: 9.479
In [70]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_series_to_compare(inv_y,inv_yhat,"Actual Price","Predicted Price", "Actual Price Versus LSTM Predicted Price")
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
7. Bench Mark Model¶
In this section we will check our bench mark model. As is proposed in my proposal my bench mark model is a simple linear regressor model.
Load the preprocessed data¶
In [34]:
from pandas import read_csv
from pandas import datetime
from pandas import DataFrame
from pandas import concat
from matplotlib import pyplot
from sklearn.metrics import mean_squared_error
from math import sqrt
# Create lagged dataset
values = pd.DataFrame(df_weekly["Settle"].values)
df_benchmark = concat([values.shift(1), values], axis=1)
df_benchmark.columns = ['t', 't+1']
display(df_benchmark.head(5))
| t | t+1 | |
|---|---|---|
| 0 | NaN | 235.50 |
| 1 | 235.50 | 228.25 |
| 2 | 228.25 | 235.50 |
| 3 | 235.50 | 241.00 |
| 4 | 241.00 | 253.50 |
In [35]:
# split into train , validation and test sets
X = df_benchmark.values
train, validation, test = X[1:validation_start], X[validation_start:testing_start],X[testing_start:]
train_bench_X, train_bench_y = train[:,0], train[:,1]
validation_bench_X, validation_bench_y = validation[:,0], validation[:,1]
test_bench_X, test_bench_y = test[:,0], test[:,1]
In [36]:
%load_ext autoreload
%autoreload 2
import models
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
In [37]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
predictions,rmse=models.make_benchmark_model_prediction(validation_bench_X,validation_bench_y)
print('Benchmark Model on Validation Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload Benchmark Model on Validation Data RMSE: 8.750
In [38]:
%load_ext autoreload
%autoreload 2
import visuals
visuals.plot_series_to_compare(validation_bench_y,predictions,"Actual Price","Predicted Price", "Actual Price Versus Benchmark Model Predicted Price")
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload
8. Test model on unseen data¶
Test LSTM model on unseen data¶
In [71]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
inv_yhat, inv_y, rmse=models.make_lstm_prediction(test_X,test_y,model,scaler)
print('LSTM Moddel on Test Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload LSTM Moddel on Test Data RMSE: 12.079
Test Benchmark model on unseen data¶
In [72]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
predictions,rmse=models.make_benchmark_model_prediction(test_bench_X,test_bench_y)
print('Benchmark Model on Test Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload Benchmark Model on Test Data RMSE: 8.293
9. Tune basic LSTM Model¶
In [41]:
%load_ext autoreload
%autoreload 2
import tune_model
tune_model.tune_memmory_cells(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use:
%reload_ext autoreload
>1/5 param=1.000000, loss=0.012129
>2/5 param=1.000000, loss=0.014510
>3/5 param=1.000000, loss=0.011251
>4/5 param=1.000000, loss=0.013165
>5/5 param=1.000000, loss=0.012103
>1/5 param=5.000000, loss=0.011613
>2/5 param=5.000000, loss=0.012066
>3/5 param=5.000000, loss=0.011987
>4/5 param=5.000000, loss=0.012024
>5/5 param=5.000000, loss=0.012577
>1/5 param=10.000000, loss=0.012330
>2/5 param=10.000000, loss=0.013115
>3/5 param=10.000000, loss=0.013052
>4/5 param=10.000000, loss=0.011792
>5/5 param=10.000000, loss=0.013219
>1/5 param=25.000000, loss=0.011451
>2/5 param=25.000000, loss=0.013046
>3/5 param=25.000000, loss=0.011217
>4/5 param=25.000000, loss=0.011381
>5/5 param=25.000000, loss=0.011058
>1/5 param=50.000000, loss=0.012644
>2/5 param=50.000000, loss=0.012646
>3/5 param=50.000000, loss=0.011140
>4/5 param=50.000000, loss=0.013345
>5/5 param=50.000000, loss=0.012515
>1/5 param=100.000000, loss=0.011604
>2/5 param=100.000000, loss=0.015141
>3/5 param=100.000000, loss=0.012493
>4/5 param=100.000000, loss=0.012222
>5/5 param=100.000000, loss=0.013316
>1/5 param=200.000000, loss=0.011432
>2/5 param=200.000000, loss=0.012943
>3/5 param=200.000000, loss=0.010766
>4/5 param=200.000000, loss=0.014985
>5/5 param=200.000000, loss=0.011893
1 5 10 25 50 100 200
count 5.000000 5.000000 5.000000 5.000000 5.000000 5.000000 5.000000
mean 0.012632 0.012053 0.012702 0.011631 0.012458 0.012955 0.012404
std 0.001250 0.000344 0.000618 0.000806 0.000805 0.001368 0.001646
min 0.011251 0.011613 0.011792 0.011058 0.011140 0.011604 0.010766
25% 0.012103 0.011987 0.012330 0.011217 0.012515 0.012222 0.011432
50% 0.012129 0.012024 0.013052 0.011381 0.012644 0.012493 0.011893
75% 0.013165 0.012066 0.013115 0.011451 0.012646 0.013316 0.012943
max 0.014510 0.012577 0.013219 0.013046 0.013345 0.015141 0.014985
In [42]:
%load_ext autoreload
%autoreload 2
import tune_model
tune_model.tune_batch_size(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use:
%reload_ext autoreload
>1/5 param=2.000000, loss=0.017062
>2/5 param=2.000000, loss=0.017288
>3/5 param=2.000000, loss=0.019628
>4/5 param=2.000000, loss=0.017816
>5/5 param=2.000000, loss=0.019510
>1/5 param=4.000000, loss=0.012946
>2/5 param=4.000000, loss=0.013468
>3/5 param=4.000000, loss=0.012065
>4/5 param=4.000000, loss=0.012588
>5/5 param=4.000000, loss=0.013342
>1/5 param=8.000000, loss=0.014913
>2/5 param=8.000000, loss=0.016104
>3/5 param=8.000000, loss=0.015724
>4/5 param=8.000000, loss=0.015701
>5/5 param=8.000000, loss=0.014112
>1/5 param=32.000000, loss=0.011369
>2/5 param=32.000000, loss=0.011775
>3/5 param=32.000000, loss=0.012824
>4/5 param=32.000000, loss=0.012704
>5/5 param=32.000000, loss=0.011133
>1/5 param=64.000000, loss=0.011609
>2/5 param=64.000000, loss=0.011532
>3/5 param=64.000000, loss=0.013435
>4/5 param=64.000000, loss=0.011951
>5/5 param=64.000000, loss=0.012349
>1/5 param=128.000000, loss=0.011928
>2/5 param=128.000000, loss=0.012988
>3/5 param=128.000000, loss=0.011940
>4/5 param=128.000000, loss=0.011974
>5/5 param=128.000000, loss=0.011488
>1/5 param=256.000000, loss=0.011780
>2/5 param=256.000000, loss=0.013215
>3/5 param=256.000000, loss=0.012355
>4/5 param=256.000000, loss=0.011390
>5/5 param=256.000000, loss=0.011595
2 4 8 32 64 128 256
count 5.000000 5.000000 5.000000 5.000000 5.000000 5.000000 5.000000
mean 0.018261 0.012882 0.015311 0.011961 0.012175 0.012064 0.012067
std 0.001226 0.000573 0.000798 0.000769 0.000775 0.000554 0.000736
min 0.017062 0.012065 0.014112 0.011133 0.011532 0.011488 0.011390
25% 0.017288 0.012588 0.014913 0.011369 0.011609 0.011928 0.011595
50% 0.017816 0.012946 0.015701 0.011775 0.011951 0.011940 0.011780
75% 0.019510 0.013342 0.015724 0.012704 0.012349 0.011974 0.012355
max 0.019628 0.013468 0.016104 0.012824 0.013435 0.012988 0.013215
In [85]:
%load_ext autoreload
%autoreload 2
import tune_model
tune_model.tune_learning_rate(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use:
%reload_ext autoreload
>1/5 param=0.100000, loss=0.011195
>2/5 param=0.100000, loss=0.011809
>3/5 param=0.100000, loss=0.018709
>4/5 param=0.100000, loss=0.032624
>5/5 param=0.100000, loss=0.015232
>1/5 param=0.001000, loss=0.011950
>2/5 param=0.001000, loss=0.012022
>3/5 param=0.001000, loss=0.012852
>4/5 param=0.001000, loss=0.012499
>5/5 param=0.001000, loss=0.011699
>1/5 param=0.000100, loss=0.033569
>2/5 param=0.000100, loss=0.027969
>3/5 param=0.000100, loss=0.051960
>4/5 param=0.000100, loss=0.043928
>5/5 param=0.000100, loss=0.035140
0.1 0.001 0.0001
count 5.000000 5.000000 5.000000
mean 0.017914 0.012204 0.038513
std 0.008756 0.000464 0.009449
min 0.011195 0.011699 0.027969
25% 0.011809 0.011950 0.033569
50% 0.015232 0.012022 0.035140
75% 0.018709 0.012499 0.043928
max 0.032624 0.012852 0.051960
In [86]:
%load_ext autoreload
%autoreload 2
import tune_model
tune_model.tune_weight_regularization(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use:
%reload_ext autoreload
>1/5 param=1.000000, loss=0.017913
>2/5 param=1.000000, loss=0.017820
>3/5 param=1.000000, loss=0.017644
>4/5 param=1.000000, loss=0.018630
>5/5 param=1.000000, loss=0.018884
>1/5 param=2.000000, loss=0.033898
>2/5 param=2.000000, loss=0.035481
>3/5 param=2.000000, loss=0.036565
>4/5 param=2.000000, loss=0.036193
>5/5 param=2.000000, loss=0.035109
>1/5 param=3.000000, loss=0.012555
>2/5 param=3.000000, loss=0.011829
>3/5 param=3.000000, loss=0.012490
>4/5 param=3.000000, loss=0.011938
>5/5 param=3.000000, loss=0.011889
>1/5 param=4.000000, loss=0.037587
>2/5 param=4.000000, loss=0.039391
>3/5 param=4.000000, loss=0.038415
>4/5 param=4.000000, loss=0.039397
>5/5 param=4.000000, loss=0.038773
1 2 3 4
count 5.000000 5.000000 5.000000 5.000000
mean 0.018178 0.035449 0.012140 0.038713
std 0.000544 0.001040 0.000352 0.000756
min 0.017644 0.033898 0.011829 0.037587
25% 0.017820 0.035109 0.011889 0.038415
50% 0.017913 0.035481 0.011938 0.038773
75% 0.018630 0.036193 0.012490 0.039391
max 0.018884 0.036565 0.012555 0.039397
10. Test Improved LSTM Model¶
In [173]:
%load_ext autoreload
%autoreload 2
import models
model,history=models.improved_lstm_model(train_X,train_y,validation_X,validation_y)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload Train on 550 samples, validate on 53 samples Epoch 1/500 - 28s - loss: 1.1178 - val_loss: 0.8899 Epoch 2/500 - 0s - loss: 1.0660 - val_loss: 0.8304 Epoch 3/500 - 0s - loss: 1.0112 - val_loss: 0.7748 Epoch 4/500 - 0s - loss: 0.9573 - val_loss: 0.7582 Epoch 5/500 - 0s - loss: 0.9143 - val_loss: 0.7839 Epoch 6/500 - 0s - loss: 0.8879 - val_loss: 0.8102 Epoch 7/500 - 0s - loss: 0.8721 - val_loss: 0.8232 Epoch 8/500 - 0s - loss: 0.8608 - val_loss: 0.8247 Epoch 9/500 - 0s - loss: 0.8509 - val_loss: 0.8184 Epoch 10/500 - 0s - loss: 0.8417 - val_loss: 0.8072 Epoch 11/500 - 0s - loss: 0.8328 - val_loss: 0.7936 Epoch 12/500 - 0s - loss: 0.8241 - val_loss: 0.7790 Epoch 13/500 - 0s - loss: 0.8158 - val_loss: 0.7646 Epoch 14/500 - 0s - loss: 0.8076 - val_loss: 0.7510 Epoch 15/500 - 0s - loss: 0.7994 - val_loss: 0.7384 Epoch 16/500 - 0s - loss: 0.7911 - val_loss: 0.7266 Epoch 17/500 - 0s - loss: 0.7828 - val_loss: 0.7157 Epoch 18/500 - 0s - loss: 0.7745 - val_loss: 0.7062 Epoch 19/500 - 0s - loss: 0.7660 - val_loss: 0.6976 Epoch 20/500 - 0s - loss: 0.7574 - val_loss: 0.6894 Epoch 21/500 - 0s - loss: 0.7487 - val_loss: 0.6814 Epoch 22/500 - 0s - loss: 0.7401 - val_loss: 0.6738 Epoch 23/500 - 0s - loss: 0.7314 - val_loss: 0.6663 Epoch 24/500 - 0s - loss: 0.7227 - val_loss: 0.6589 Epoch 25/500 - 0s - loss: 0.7139 - val_loss: 0.6515 Epoch 26/500 - 0s - loss: 0.7052 - val_loss: 0.6437 Epoch 27/500 - 0s - loss: 0.6966 - val_loss: 0.6356 Epoch 28/500 - 0s - loss: 0.6880 - val_loss: 0.6271 Epoch 29/500 - 0s - loss: 0.6795 - val_loss: 0.6186 Epoch 30/500 - 0s - loss: 0.6711 - val_loss: 0.6100 Epoch 31/500 - 0s - loss: 0.6626 - val_loss: 0.6013 Epoch 32/500 - 0s - loss: 0.6541 - val_loss: 0.5929 Epoch 33/500 - 0s - loss: 0.6456 - val_loss: 0.5849 Epoch 34/500 - 0s - loss: 0.6370 - val_loss: 0.5772 Epoch 35/500 - 0s - loss: 0.6283 - val_loss: 0.5696 Epoch 36/500 - 0s - loss: 0.6196 - val_loss: 0.5621 Epoch 37/500 - 0s - loss: 0.6109 - val_loss: 0.5544 Epoch 38/500 - 0s - loss: 0.6021 - val_loss: 0.5465 Epoch 39/500 - 0s - loss: 0.5934 - val_loss: 0.5383 Epoch 40/500 - 0s - loss: 0.5847 - val_loss: 0.5299 Epoch 41/500 - 0s - loss: 0.5760 - val_loss: 0.5214 Epoch 42/500 - 0s - loss: 0.5672 - val_loss: 0.5134 Epoch 43/500 - 0s - loss: 0.5584 - val_loss: 0.5058 Epoch 44/500 - 0s - loss: 0.5495 - val_loss: 0.4984 Epoch 45/500 - 0s - loss: 0.5405 - val_loss: 0.4913 Epoch 46/500 - 0s - loss: 0.5318 - val_loss: 0.4844 Epoch 47/500 - 0s - loss: 0.5231 - val_loss: 0.4776 Epoch 48/500 - 0s - loss: 0.5147 - val_loss: 0.4704 Epoch 49/500 - 0s - loss: 0.5064 - val_loss: 0.4633 Epoch 50/500 - 0s - loss: 0.4984 - val_loss: 0.4568 Epoch 51/500 - 0s - loss: 0.4907 - val_loss: 0.4517 Epoch 52/500 - 0s - loss: 0.4830 - val_loss: 0.4471 Epoch 53/500 - 0s - loss: 0.4754 - val_loss: 0.4425 Epoch 54/500 - 0s - loss: 0.4682 - val_loss: 0.4378 Epoch 55/500 - 0s - loss: 0.4612 - val_loss: 0.4334 Epoch 56/500 - 0s - loss: 0.4543 - val_loss: 0.4288 Epoch 57/500 - 0s - loss: 0.4477 - val_loss: 0.4245 Epoch 58/500 - 0s - loss: 0.4412 - val_loss: 0.4199 Epoch 59/500 - 0s - loss: 0.4349 - val_loss: 0.4151 Epoch 60/500 - 0s - loss: 0.4288 - val_loss: 0.4105 Epoch 61/500 - 0s - loss: 0.4229 - val_loss: 0.4059 Epoch 62/500 - 0s - loss: 0.4171 - val_loss: 0.4007 Epoch 63/500 - 0s - loss: 0.4115 - val_loss: 0.3955 Epoch 64/500 - 0s - loss: 0.4062 - val_loss: 0.3905 Epoch 65/500 - 0s - loss: 0.4011 - val_loss: 0.3859 Epoch 66/500 - 0s - loss: 0.3960 - val_loss: 0.3811 Epoch 67/500 - 0s - loss: 0.3910 - val_loss: 0.3763 Epoch 68/500 - 0s - loss: 0.3862 - val_loss: 0.3716 Epoch 69/500 - 0s - loss: 0.3815 - val_loss: 0.3670 Epoch 70/500 - 0s - loss: 0.3769 - val_loss: 0.3622 Epoch 71/500 - 0s - loss: 0.3724 - val_loss: 0.3574 Epoch 72/500 - 0s - loss: 0.3679 - val_loss: 0.3528 Epoch 73/500 - 0s - loss: 0.3636 - val_loss: 0.3482 Epoch 74/500 - 0s - loss: 0.3593 - val_loss: 0.3435 Epoch 75/500 - 0s - loss: 0.3551 - val_loss: 0.3392 Epoch 76/500 - 0s - loss: 0.3509 - val_loss: 0.3352 Epoch 77/500 - 0s - loss: 0.3468 - val_loss: 0.3307 Epoch 78/500 - 0s - loss: 0.3427 - val_loss: 0.3266 Epoch 79/500 - 0s - loss: 0.3387 - val_loss: 0.3226 Epoch 80/500 - 0s - loss: 0.3348 - val_loss: 0.3184 Epoch 81/500 - 0s - loss: 0.3309 - val_loss: 0.3144 Epoch 82/500 - 0s - loss: 0.3270 - val_loss: 0.3106 Epoch 83/500 - 0s - loss: 0.3232 - val_loss: 0.3065 Epoch 84/500 - 0s - loss: 0.3194 - val_loss: 0.3026 Epoch 85/500 - 0s - loss: 0.3157 - val_loss: 0.2989 Epoch 86/500 - 0s - loss: 0.3120 - val_loss: 0.2952 Epoch 87/500 - 0s - loss: 0.3083 - val_loss: 0.2913 Epoch 88/500 - 0s - loss: 0.3047 - val_loss: 0.2876 Epoch 89/500 - 0s - loss: 0.3012 - val_loss: 0.2841 Epoch 90/500 - 0s - loss: 0.2976 - val_loss: 0.2805 Epoch 91/500 - 0s - loss: 0.2942 - val_loss: 0.2770 Epoch 92/500 - 0s - loss: 0.2907 - val_loss: 0.2733 Epoch 93/500 - 0s - loss: 0.2873 - val_loss: 0.2701 Epoch 94/500 - 0s - loss: 0.2839 - val_loss: 0.2666 Epoch 95/500 - 0s - loss: 0.2805 - val_loss: 0.2633 Epoch 96/500 - 0s - loss: 0.2772 - val_loss: 0.2599 Epoch 97/500 - 0s - loss: 0.2740 - val_loss: 0.2564 Epoch 98/500 - 0s - loss: 0.2707 - val_loss: 0.2535 Epoch 99/500 - 0s - loss: 0.2675 - val_loss: 0.2501 Epoch 100/500 - 0s - loss: 0.2643 - val_loss: 0.2470 Epoch 101/500 - 0s - loss: 0.2611 - val_loss: 0.2438 Epoch 102/500 - 0s - loss: 0.2580 - val_loss: 0.2407 Epoch 103/500 - 0s - loss: 0.2549 - val_loss: 0.2376 Epoch 104/500 - 0s - loss: 0.2519 - val_loss: 0.2347 Epoch 105/500 - 0s - loss: 0.2489 - val_loss: 0.2315 Epoch 106/500 - 0s - loss: 0.2459 - val_loss: 0.2287 Epoch 107/500 - 0s - loss: 0.2429 - val_loss: 0.2256 Epoch 108/500 - 0s - loss: 0.2400 - val_loss: 0.2227 Epoch 109/500 - 0s - loss: 0.2371 - val_loss: 0.2199 Epoch 110/500 - 0s - loss: 0.2342 - val_loss: 0.2170 Epoch 111/500 - 0s - loss: 0.2314 - val_loss: 0.2143 Epoch 112/500 - 0s - loss: 0.2286 - val_loss: 0.2114 Epoch 113/500 - 0s - loss: 0.2258 - val_loss: 0.2087 Epoch 114/500 - 0s - loss: 0.2231 - val_loss: 0.2060 Epoch 115/500 - 0s - loss: 0.2204 - val_loss: 0.2033 Epoch 116/500 - 0s - loss: 0.2177 - val_loss: 0.2008 Epoch 117/500 - 0s - loss: 0.2150 - val_loss: 0.1981 Epoch 118/500 - 0s - loss: 0.2124 - val_loss: 0.1954 Epoch 119/500 - 0s - loss: 0.2098 - val_loss: 0.1930 Epoch 120/500 - 0s - loss: 0.2073 - val_loss: 0.1903 Epoch 121/500 - 0s - loss: 0.2047 - val_loss: 0.1878 Epoch 122/500 - 0s - loss: 0.2022 - val_loss: 0.1854 Epoch 123/500 - 0s - loss: 0.1997 - val_loss: 0.1829 Epoch 124/500 - 0s - loss: 0.1973 - val_loss: 0.1805 Epoch 125/500 - 0s - loss: 0.1948 - val_loss: 0.1781 Epoch 126/500 - 0s - loss: 0.1924 - val_loss: 0.1758 Epoch 127/500 - 0s - loss: 0.1901 - val_loss: 0.1735 Epoch 128/500 - 0s - loss: 0.1877 - val_loss: 0.1711 Epoch 129/500 - 0s - loss: 0.1854 - val_loss: 0.1690 Epoch 130/500 - 0s - loss: 0.1831 - val_loss: 0.1666 Epoch 131/500 - 0s - loss: 0.1808 - val_loss: 0.1642 Epoch 132/500 - 0s - loss: 0.1786 - val_loss: 0.1622 Epoch 133/500 - 0s - loss: 0.1764 - val_loss: 0.1599 Epoch 134/500 - 0s - loss: 0.1742 - val_loss: 0.1578 Epoch 135/500 - 0s - loss: 0.1720 - val_loss: 0.1557 Epoch 136/500 - 0s - loss: 0.1699 - val_loss: 0.1535 Epoch 137/500 - 0s - loss: 0.1678 - val_loss: 0.1515 Epoch 138/500 - 0s - loss: 0.1657 - val_loss: 0.1493 Epoch 139/500 - 0s - loss: 0.1636 - val_loss: 0.1475 Epoch 140/500 - 0s - loss: 0.1616 - val_loss: 0.1454 Epoch 141/500 - 0s - loss: 0.1595 - val_loss: 0.1434 Epoch 142/500 - 0s - loss: 0.1575 - val_loss: 0.1415 Epoch 143/500 - 0s - loss: 0.1556 - val_loss: 0.1395 Epoch 144/500 - 0s - loss: 0.1536 - val_loss: 0.1377 Epoch 145/500 - 0s - loss: 0.1517 - val_loss: 0.1357 Epoch 146/500 - 0s - loss: 0.1498 - val_loss: 0.1340 Epoch 147/500 - 0s - loss: 0.1479 - val_loss: 0.1320 Epoch 148/500 - 0s - loss: 0.1461 - val_loss: 0.1301 Epoch 149/500 - 0s - loss: 0.1442 - val_loss: 0.1285 Epoch 150/500 - 0s - loss: 0.1424 - val_loss: 0.1265 Epoch 151/500 - 0s - loss: 0.1406 - val_loss: 0.1250 Epoch 152/500 - 0s - loss: 0.1389 - val_loss: 0.1230 Epoch 153/500 - 0s - loss: 0.1371 - val_loss: 0.1215 Epoch 154/500 - 0s - loss: 0.1354 - val_loss: 0.1197 Epoch 155/500 - 0s - loss: 0.1337 - val_loss: 0.1181 Epoch 156/500 - 0s - loss: 0.1320 - val_loss: 0.1163 Epoch 157/500 - 0s - loss: 0.1304 - val_loss: 0.1148 Epoch 158/500 - 0s - loss: 0.1287 - val_loss: 0.1132 Epoch 159/500 - 0s - loss: 0.1271 - val_loss: 0.1115 Epoch 160/500 - 0s - loss: 0.1255 - val_loss: 0.1100 Epoch 161/500 - 0s - loss: 0.1239 - val_loss: 0.1084 Epoch 162/500 - 0s - loss: 0.1224 - val_loss: 0.1069 Epoch 163/500 - 0s - loss: 0.1209 - val_loss: 0.1054 Epoch 164/500 - 0s - loss: 0.1193 - val_loss: 0.1039 Epoch 165/500 - 0s - loss: 0.1178 - val_loss: 0.1023 Epoch 166/500 - 0s - loss: 0.1164 - val_loss: 0.1010 Epoch 167/500 - 0s - loss: 0.1149 - val_loss: 0.0995 Epoch 168/500 - 0s - loss: 0.1135 - val_loss: 0.0981 Epoch 169/500 - 0s - loss: 0.1120 - val_loss: 0.0968 Epoch 170/500 - 0s - loss: 0.1107 - val_loss: 0.0953 Epoch 171/500 - 0s - loss: 0.1093 - val_loss: 0.0941 Epoch 172/500 - 0s - loss: 0.1079 - val_loss: 0.0926 Epoch 173/500 - 0s - loss: 0.1065 - val_loss: 0.0913 Epoch 174/500 - 0s - loss: 0.1052 - val_loss: 0.0900 Epoch 175/500 - 0s - loss: 0.1039 - val_loss: 0.0887 Epoch 176/500 - 0s - loss: 0.1026 - val_loss: 0.0874 Epoch 177/500 - 0s - loss: 0.1013 - val_loss: 0.0863 Epoch 178/500 - 0s - loss: 0.1001 - val_loss: 0.0848 Epoch 179/500 - 0s - loss: 0.0988 - val_loss: 0.0838 Epoch 180/500 - 0s - loss: 0.0976 - val_loss: 0.0824 Epoch 181/500 - 0s - loss: 0.0964 - val_loss: 0.0814 Epoch 182/500 - 0s - loss: 0.0952 - val_loss: 0.0801 Epoch 183/500 - 0s - loss: 0.0940 - val_loss: 0.0792 Epoch 184/500 - 0s - loss: 0.0929 - val_loss: 0.0778 Epoch 185/500 - 0s - loss: 0.0917 - val_loss: 0.0769 Epoch 186/500 - 0s - loss: 0.0906 - val_loss: 0.0756 Epoch 187/500 - 0s - loss: 0.0895 - val_loss: 0.0746 Epoch 188/500 - 0s - loss: 0.0884 - val_loss: 0.0734 Epoch 189/500 - 0s - loss: 0.0873 - val_loss: 0.0726 Epoch 190/500 - 0s - loss: 0.0863 - val_loss: 0.0712 Epoch 191/500 - 0s - loss: 0.0852 - val_loss: 0.0704 Epoch 192/500 - 0s - loss: 0.0842 - val_loss: 0.0693 Epoch 193/500 - 0s - loss: 0.0832 - val_loss: 0.0681 Epoch 194/500 - 0s - loss: 0.0822 - val_loss: 0.0673 Epoch 195/500 - 0s - loss: 0.0812 - val_loss: 0.0662 Epoch 196/500 - 0s - loss: 0.0802 - val_loss: 0.0655 Epoch 197/500 - 0s - loss: 0.0792 - val_loss: 0.0644 Epoch 198/500 - 0s - loss: 0.0783 - val_loss: 0.0636 Epoch 199/500 - 0s - loss: 0.0773 - val_loss: 0.0626 Epoch 200/500 - 0s - loss: 0.0764 - val_loss: 0.0619 Epoch 201/500 - 0s - loss: 0.0755 - val_loss: 0.0607 Epoch 202/500 - 0s - loss: 0.0746 - val_loss: 0.0599 Epoch 203/500 - 0s - loss: 0.0737 - val_loss: 0.0590 Epoch 204/500 - 0s - loss: 0.0729 - val_loss: 0.0579 Epoch 205/500 - 0s - loss: 0.0720 - val_loss: 0.0574 Epoch 206/500 - 0s - loss: 0.0712 - val_loss: 0.0565 Epoch 207/500 - 0s - loss: 0.0703 - val_loss: 0.0557 Epoch 208/500 - 0s - loss: 0.0695 - val_loss: 0.0549 Epoch 209/500 - 0s - loss: 0.0687 - val_loss: 0.0541 Epoch 210/500 - 0s - loss: 0.0679 - val_loss: 0.0536 Epoch 211/500 - 0s - loss: 0.0672 - val_loss: 0.0524 Epoch 212/500 - 0s - loss: 0.0664 - val_loss: 0.0519 Epoch 213/500 - 0s - loss: 0.0656 - val_loss: 0.0511 Epoch 214/500 - 0s - loss: 0.0649 - val_loss: 0.0504 Epoch 215/500 - 0s - loss: 0.0642 - val_loss: 0.0497 Epoch 216/500 - 0s - loss: 0.0634 - val_loss: 0.0489 Epoch 217/500 - 0s - loss: 0.0627 - val_loss: 0.0484 Epoch 218/500 - 0s - loss: 0.0620 - val_loss: 0.0474 Epoch 219/500 - 0s - loss: 0.0613 - val_loss: 0.0469 Epoch 220/500 - 0s - loss: 0.0607 - val_loss: 0.0461 Epoch 221/500 - 0s - loss: 0.0600 - val_loss: 0.0456 Epoch 222/500 - 0s - loss: 0.0593 - val_loss: 0.0449 Epoch 223/500 - 0s - loss: 0.0587 - val_loss: 0.0442 Epoch 224/500 - 0s - loss: 0.0580 - val_loss: 0.0437 Epoch 225/500 - 0s - loss: 0.0574 - val_loss: 0.0430 Epoch 226/500 - 0s - loss: 0.0568 - val_loss: 0.0424 Epoch 227/500 - 0s - loss: 0.0562 - val_loss: 0.0418 Epoch 228/500 - 0s - loss: 0.0556 - val_loss: 0.0412 Epoch 229/500 - 0s - loss: 0.0550 - val_loss: 0.0407 Epoch 230/500 - 0s - loss: 0.0544 - val_loss: 0.0400 Epoch 231/500 - 0s - loss: 0.0539 - val_loss: 0.0396 Epoch 232/500 - 0s - loss: 0.0533 - val_loss: 0.0390 Epoch 233/500 - 0s - loss: 0.0528 - val_loss: 0.0384 Epoch 234/500 - 0s - loss: 0.0522 - val_loss: 0.0382 Epoch 235/500 - 0s - loss: 0.0517 - val_loss: 0.0371 Epoch 236/500 - 0s - loss: 0.0512 - val_loss: 0.0370 Epoch 237/500 - 0s - loss: 0.0507 - val_loss: 0.0362 Epoch 238/500 - 0s - loss: 0.0502 - val_loss: 0.0360 Epoch 239/500 - 0s - loss: 0.0497 - val_loss: 0.0353 Epoch 240/500 - 0s - loss: 0.0492 - val_loss: 0.0351 Epoch 241/500 - 0s - loss: 0.0487 - val_loss: 0.0345 Epoch 242/500 - 0s - loss: 0.0482 - val_loss: 0.0341 Epoch 243/500 - 0s - loss: 0.0478 - val_loss: 0.0338 Epoch 244/500 - 0s - loss: 0.0473 - val_loss: 0.0330 Epoch 245/500 - 0s - loss: 0.0469 - val_loss: 0.0330 Epoch 246/500 - 0s - loss: 0.0465 - val_loss: 0.0321 Epoch 247/500 - 0s - loss: 0.0460 - val_loss: 0.0321 Epoch 248/500 - 0s - loss: 0.0456 - val_loss: 0.0314 Epoch 249/500 - 0s - loss: 0.0452 - val_loss: 0.0313 Epoch 250/500 - 0s - loss: 0.0448 - val_loss: 0.0306 Epoch 251/500 - 0s - loss: 0.0444 - val_loss: 0.0304 Epoch 252/500 - 0s - loss: 0.0440 - val_loss: 0.0298 Epoch 253/500 - 0s - loss: 0.0436 - val_loss: 0.0295 Epoch 254/500 - 0s - loss: 0.0432 - val_loss: 0.0291 Epoch 255/500 - 0s - loss: 0.0428 - val_loss: 0.0287 Epoch 256/500 - 0s - loss: 0.0424 - val_loss: 0.0284 Epoch 257/500 - 0s - loss: 0.0421 - val_loss: 0.0280 Epoch 258/500 - 0s - loss: 0.0417 - val_loss: 0.0276 Epoch 259/500 - 0s - loss: 0.0414 - val_loss: 0.0273 Epoch 260/500 - 0s - loss: 0.0410 - val_loss: 0.0270 Epoch 261/500 - 0s - loss: 0.0407 - val_loss: 0.0267 Epoch 262/500 - 0s - loss: 0.0404 - val_loss: 0.0263 Epoch 263/500 - 0s - loss: 0.0401 - val_loss: 0.0260 Epoch 264/500 - 0s - loss: 0.0397 - val_loss: 0.0257 Epoch 265/500 - 0s - loss: 0.0394 - val_loss: 0.0255 Epoch 266/500 - 0s - loss: 0.0391 - val_loss: 0.0252 Epoch 267/500 - 0s - loss: 0.0388 - val_loss: 0.0248 Epoch 268/500 - 0s - loss: 0.0385 - val_loss: 0.0248 Epoch 269/500 - 0s - loss: 0.0383 - val_loss: 0.0242 Epoch 270/500 - 0s - loss: 0.0380 - val_loss: 0.0243 Epoch 271/500 - 0s - loss: 0.0377 - val_loss: 0.0236 Epoch 272/500 - 0s - loss: 0.0374 - val_loss: 0.0237 Epoch 273/500 - 0s - loss: 0.0372 - val_loss: 0.0231 Epoch 274/500 - 0s - loss: 0.0369 - val_loss: 0.0230 Epoch 275/500 - 0s - loss: 0.0366 - val_loss: 0.0227 Epoch 276/500 - 0s - loss: 0.0364 - val_loss: 0.0225 Epoch 277/500 - 0s - loss: 0.0361 - val_loss: 0.0224 Epoch 278/500 - 0s - loss: 0.0359 - val_loss: 0.0220 Epoch 279/500 - 0s - loss: 0.0357 - val_loss: 0.0217 Epoch 280/500 - 0s - loss: 0.0354 - val_loss: 0.0217 Epoch 281/500 - 0s - loss: 0.0352 - val_loss: 0.0213 Epoch 282/500 - 0s - loss: 0.0350 - val_loss: 0.0213 Epoch 283/500 - 0s - loss: 0.0348 - val_loss: 0.0211 Epoch 284/500 - 0s - loss: 0.0346 - val_loss: 0.0207 Epoch 285/500 - 0s - loss: 0.0343 - val_loss: 0.0208 Epoch 286/500 - 0s - loss: 0.0341 - val_loss: 0.0204 Epoch 287/500 - 0s - loss: 0.0339 - val_loss: 0.0202 Epoch 288/500 - 0s - loss: 0.0337 - val_loss: 0.0201 Epoch 289/500 - 0s - loss: 0.0335 - val_loss: 0.0198 Epoch 290/500 - 0s - loss: 0.0333 - val_loss: 0.0197 Epoch 291/500 - 0s - loss: 0.0332 - val_loss: 0.0196 Epoch 292/500 - 0s - loss: 0.0330 - val_loss: 0.0190 Epoch 293/500 - 0s - loss: 0.0329 - val_loss: 0.0192 Epoch 294/500 - 0s - loss: 0.0327 - val_loss: 0.0186 Epoch 295/500 - 0s - loss: 0.0325 - val_loss: 0.0190 Epoch 296/500 - 0s - loss: 0.0323 - val_loss: 0.0183 Epoch 297/500 - 0s - loss: 0.0321 - val_loss: 0.0186 Epoch 298/500 - 0s - loss: 0.0320 - val_loss: 0.0181 Epoch 299/500 - 0s - loss: 0.0318 - val_loss: 0.0185 Epoch 300/500 - 0s - loss: 0.0317 - val_loss: 0.0177 Epoch 301/500 - 0s - loss: 0.0315 - val_loss: 0.0180 Epoch 302/500 - 0s - loss: 0.0314 - val_loss: 0.0175 Epoch 303/500 - 0s - loss: 0.0312 - val_loss: 0.0174 Epoch 304/500 - 0s - loss: 0.0311 - val_loss: 0.0173 Epoch 305/500 - 0s - loss: 0.0309 - val_loss: 0.0174 Epoch 306/500 - 0s - loss: 0.0308 - val_loss: 0.0172 Epoch 307/500 - 0s - loss: 0.0307 - val_loss: 0.0172 Epoch 308/500 - 0s - loss: 0.0306 - val_loss: 0.0169 Epoch 309/500 - 0s - loss: 0.0304 - val_loss: 0.0169 Epoch 310/500 - 0s - loss: 0.0303 - val_loss: 0.0167 Epoch 311/500 - 0s - loss: 0.0302 - val_loss: 0.0164 Epoch 312/500 - 0s - loss: 0.0300 - val_loss: 0.0165 Epoch 313/500 - 0s - loss: 0.0299 - val_loss: 0.0165 Epoch 314/500 - 0s - loss: 0.0298 - val_loss: 0.0161 Epoch 315/500 - 0s - loss: 0.0297 - val_loss: 0.0161 Epoch 316/500 - 0s - loss: 0.0296 - val_loss: 0.0159 Epoch 317/500 - 0s - loss: 0.0295 - val_loss: 0.0158 Epoch 318/500 - 0s - loss: 0.0294 - val_loss: 0.0159 Epoch 319/500 - 0s - loss: 0.0293 - val_loss: 0.0156 Epoch 320/500 - 0s - loss: 0.0292 - val_loss: 0.0158 Epoch 321/500 - 0s - loss: 0.0291 - val_loss: 0.0155 Epoch 322/500 - 0s - loss: 0.0290 - val_loss: 0.0154 Epoch 323/500 - 0s - loss: 0.0289 - val_loss: 0.0155 Epoch 324/500 - 0s - loss: 0.0288 - val_loss: 0.0152 Epoch 325/500 - 0s - loss: 0.0287 - val_loss: 0.0153 Epoch 326/500 - 0s - loss: 0.0286 - val_loss: 0.0152 Epoch 327/500 - 0s - loss: 0.0286 - val_loss: 0.0149 Epoch 328/500 - 0s - loss: 0.0285 - val_loss: 0.0149 Epoch 329/500 - 0s - loss: 0.0284 - val_loss: 0.0152 Epoch 330/500 - 0s - loss: 0.0284 - val_loss: 0.0145 Epoch 331/500 - 0s - loss: 0.0282 - val_loss: 0.0148 Epoch 332/500 - 0s - loss: 0.0282 - val_loss: 0.0146 Epoch 333/500 - 0s - loss: 0.0281 - val_loss: 0.0144 Epoch 334/500 - 0s - loss: 0.0280 - val_loss: 0.0146 Epoch 335/500 - 0s - loss: 0.0280 - val_loss: 0.0143 Epoch 336/500 - 0s - loss: 0.0279 - val_loss: 0.0145 Epoch 337/500 - 0s - loss: 0.0278 - val_loss: 0.0142 Epoch 338/500 - 0s - loss: 0.0278 - val_loss: 0.0141 Epoch 339/500 - 0s - loss: 0.0277 - val_loss: 0.0142 Epoch 340/500 - 0s - loss: 0.0276 - val_loss: 0.0142 Epoch 341/500 - 0s - loss: 0.0276 - val_loss: 0.0140 Epoch 342/500 - 0s - loss: 0.0275 - val_loss: 0.0139 Epoch 343/500 - 0s - loss: 0.0275 - val_loss: 0.0139 Epoch 344/500 - 0s - loss: 0.0274 - val_loss: 0.0137 Epoch 345/500 - 0s - loss: 0.0273 - val_loss: 0.0140 Epoch 346/500 - 0s - loss: 0.0273 - val_loss: 0.0137 Epoch 347/500 - 0s - loss: 0.0272 - val_loss: 0.0139 Epoch 348/500 - 0s - loss: 0.0272 - val_loss: 0.0138 Epoch 349/500 - 0s - loss: 0.0272 - val_loss: 0.0135 Epoch 350/500 - 0s - loss: 0.0271 - val_loss: 0.0137 Epoch 351/500 - 0s - loss: 0.0271 - val_loss: 0.0135 Epoch 352/500 - 0s - loss: 0.0270 - val_loss: 0.0133 Epoch 353/500 - 0s - loss: 0.0270 - val_loss: 0.0135 Epoch 354/500 - 0s - loss: 0.0270 - val_loss: 0.0131 Epoch 355/500 - 0s - loss: 0.0269 - val_loss: 0.0135 Epoch 356/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 357/500 - 0s - loss: 0.0268 - val_loss: 0.0133 Epoch 358/500 - 0s - loss: 0.0268 - val_loss: 0.0131 Epoch 359/500 - 0s - loss: 0.0267 - val_loss: 0.0133 Epoch 360/500 - 0s - loss: 0.0267 - val_loss: 0.0131 Epoch 361/500 - 0s - loss: 0.0266 - val_loss: 0.0130 Epoch 362/500 - 0s - loss: 0.0266 - val_loss: 0.0130 Epoch 363/500 - 0s - loss: 0.0266 - val_loss: 0.0129 Epoch 364/500 - 0s - loss: 0.0265 - val_loss: 0.0130 Epoch 365/500 - 0s - loss: 0.0265 - val_loss: 0.0131 Epoch 366/500 - 0s - loss: 0.0265 - val_loss: 0.0128 Epoch 367/500 - 0s - loss: 0.0265 - val_loss: 0.0131 Epoch 368/500 - 0s - loss: 0.0265 - val_loss: 0.0126 Epoch 369/500 - 0s - loss: 0.0264 - val_loss: 0.0131 Epoch 370/500 - 0s - loss: 0.0264 - val_loss: 0.0130 Epoch 371/500 - 0s - loss: 0.0263 - val_loss: 0.0127 Epoch 372/500 - 0s - loss: 0.0263 - val_loss: 0.0128 Epoch 373/500 - 0s - loss: 0.0263 - val_loss: 0.0125 Epoch 374/500 - 0s - loss: 0.0262 - val_loss: 0.0128 Epoch 375/500 - 0s - loss: 0.0262 - val_loss: 0.0127 Epoch 376/500 - 0s - loss: 0.0262 - val_loss: 0.0128 Epoch 377/500 - 0s - loss: 0.0262 - val_loss: 0.0127 Epoch 378/500 - 0s - loss: 0.0261 - val_loss: 0.0126 Epoch 379/500 - 0s - loss: 0.0261 - val_loss: 0.0126 Epoch 380/500 - 0s - loss: 0.0261 - val_loss: 0.0126 Epoch 381/500 - 0s - loss: 0.0261 - val_loss: 0.0126 Epoch 382/500 - 0s - loss: 0.0260 - val_loss: 0.0125 Epoch 383/500 - 0s - loss: 0.0260 - val_loss: 0.0124 Epoch 384/500 - 0s - loss: 0.0260 - val_loss: 0.0124 Epoch 385/500 - 0s - loss: 0.0260 - val_loss: 0.0124 Epoch 386/500 - 0s - loss: 0.0260 - val_loss: 0.0126 Epoch 387/500 - 0s - loss: 0.0260 - val_loss: 0.0124 Epoch 388/500 - 0s - loss: 0.0259 - val_loss: 0.0124 Epoch 389/500 - 0s - loss: 0.0259 - val_loss: 0.0124 Epoch 390/500 - 0s - loss: 0.0259 - val_loss: 0.0123 Epoch 391/500 - 0s - loss: 0.0259 - val_loss: 0.0125 Epoch 392/500 - 0s - loss: 0.0259 - val_loss: 0.0123 Epoch 393/500 - 0s - loss: 0.0259 - val_loss: 0.0125 Epoch 394/500 - 0s - loss: 0.0258 - val_loss: 0.0123 Epoch 395/500 - 0s - loss: 0.0258 - val_loss: 0.0123 Epoch 396/500 - 0s - loss: 0.0259 - val_loss: 0.0122 Epoch 397/500 - 0s - loss: 0.0258 - val_loss: 0.0123 Epoch 398/500 - 0s - loss: 0.0258 - val_loss: 0.0124 Epoch 399/500 - 0s - loss: 0.0258 - val_loss: 0.0120 Epoch 400/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 401/500 - 0s - loss: 0.0257 - val_loss: 0.0123 Epoch 402/500 - 0s - loss: 0.0258 - val_loss: 0.0120 Epoch 403/500 - 0s - loss: 0.0257 - val_loss: 0.0123 Epoch 404/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 405/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 406/500 - 0s - loss: 0.0257 - val_loss: 0.0121 Epoch 407/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 408/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 409/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 410/500 - 0s - loss: 0.0256 - val_loss: 0.0121 Epoch 411/500 - 0s - loss: 0.0256 - val_loss: 0.0122 Epoch 412/500 - 0s - loss: 0.0256 - val_loss: 0.0121 Epoch 413/500 - 0s - loss: 0.0256 - val_loss: 0.0122 Epoch 414/500 - 0s - loss: 0.0256 - val_loss: 0.0121 Epoch 415/500 - 0s - loss: 0.0256 - val_loss: 0.0122 Epoch 416/500 - 0s - loss: 0.0256 - val_loss: 0.0120 Epoch 417/500 - 0s - loss: 0.0256 - val_loss: 0.0121 Epoch 418/500 - 0s - loss: 0.0255 - val_loss: 0.0122 Epoch 419/500 - 0s - loss: 0.0256 - val_loss: 0.0121 Epoch 420/500 - 0s - loss: 0.0256 - val_loss: 0.0119 Epoch 421/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 422/500 - 0s - loss: 0.0255 - val_loss: 0.0121 Epoch 423/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 424/500 - 0s - loss: 0.0255 - val_loss: 0.0121 Epoch 425/500 - 0s - loss: 0.0256 - val_loss: 0.0119 Epoch 426/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 427/500 - 0s - loss: 0.0257 - val_loss: 0.0124 Epoch 428/500 - 0s - loss: 0.0257 - val_loss: 0.0117 Epoch 429/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 430/500 - 0s - loss: 0.0256 - val_loss: 0.0124 Epoch 431/500 - 0s - loss: 0.0256 - val_loss: 0.0117 Epoch 432/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 433/500 - 0s - loss: 0.0255 - val_loss: 0.0121 Epoch 434/500 - 0s - loss: 0.0256 - val_loss: 0.0119 Epoch 435/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 436/500 - 0s - loss: 0.0255 - val_loss: 0.0123 Epoch 437/500 - 0s - loss: 0.0256 - val_loss: 0.0119 Epoch 438/500 - 0s - loss: 0.0256 - val_loss: 0.0118 Epoch 439/500 - 0s - loss: 0.0254 - val_loss: 0.0120 Epoch 440/500 - 0s - loss: 0.0256 - val_loss: 0.0122 Epoch 441/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 442/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 443/500 - 0s - loss: 0.0254 - val_loss: 0.0120 Epoch 444/500 - 0s - loss: 0.0255 - val_loss: 0.0119 Epoch 445/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 446/500 - 0s - loss: 0.0255 - val_loss: 0.0124 Epoch 447/500 - 0s - loss: 0.0255 - val_loss: 0.0119 Epoch 448/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 449/500 - 0s - loss: 0.0254 - val_loss: 0.0122 Epoch 450/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 451/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 452/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 453/500 - 0s - loss: 0.0254 - val_loss: 0.0120 Epoch 454/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 455/500 - 0s - loss: 0.0255 - val_loss: 0.0122 Epoch 456/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 457/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 458/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 459/500 - 0s - loss: 0.0254 - val_loss: 0.0121 Epoch 460/500 - 0s - loss: 0.0255 - val_loss: 0.0119 Epoch 461/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 462/500 - 0s - loss: 0.0254 - val_loss: 0.0122 Epoch 463/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 464/500 - 0s - loss: 0.0255 - val_loss: 0.0118 Epoch 465/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 466/500 - 0s - loss: 0.0253 - val_loss: 0.0118 Epoch 467/500 - 0s - loss: 0.0257 - val_loss: 0.0122 Epoch 468/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 469/500 - 0s - loss: 0.0253 - val_loss: 0.0118 Epoch 470/500 - 0s - loss: 0.0253 - val_loss: 0.0119 Epoch 471/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 472/500 - 0s - loss: 0.0255 - val_loss: 0.0118 Epoch 473/500 - 0s - loss: 0.0254 - val_loss: 0.0116 Epoch 474/500 - 0s - loss: 0.0254 - val_loss: 0.0124 Epoch 475/500 - 0s - loss: 0.0255 - val_loss: 0.0118 Epoch 476/500 - 0s - loss: 0.0254 - val_loss: 0.0116 Epoch 477/500 - 0s - loss: 0.0254 - val_loss: 0.0122 Epoch 478/500 - 0s - loss: 0.0255 - val_loss: 0.0117 Epoch 479/500 - 0s - loss: 0.0254 - val_loss: 0.0116 Epoch 480/500 - 0s - loss: 0.0254 - val_loss: 0.0121 Epoch 481/500 - 0s - loss: 0.0254 - val_loss: 0.0120 Epoch 482/500 - 0s - loss: 0.0255 - val_loss: 0.0116 Epoch 483/500 - 0s - loss: 0.0254 - val_loss: 0.0120 Epoch 484/500 - 0s - loss: 0.0254 - val_loss: 0.0118 Epoch 485/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 486/500 - 0s - loss: 0.0254 - val_loss: 0.0117 Epoch 487/500 - 0s - loss: 0.0253 - val_loss: 0.0120 Epoch 488/500 - 0s - loss: 0.0255 - val_loss: 0.0120 Epoch 489/500 - 0s - loss: 0.0254 - val_loss: 0.0116 Epoch 490/500 - 0s - loss: 0.0253 - val_loss: 0.0116 Epoch 491/500 - 0s - loss: 0.0254 - val_loss: 0.0121 Epoch 492/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 493/500 - 0s - loss: 0.0254 - val_loss: 0.0116 Epoch 494/500 - 0s - loss: 0.0253 - val_loss: 0.0121 Epoch 495/500 - 0s - loss: 0.0254 - val_loss: 0.0119 Epoch 496/500 - 0s - loss: 0.0255 - val_loss: 0.0116 Epoch 497/500 - 0s - loss: 0.0253 - val_loss: 0.0118 Epoch 498/500 - 0s - loss: 0.0254 - val_loss: 0.0121 Epoch 499/500 - 0s - loss: 0.0254 - val_loss: 0.0117 Epoch 500/500 - 0s - loss: 0.0254 - val_loss: 0.0117
In [174]:
pyplot.plot(history['loss'], label='train')
pyplot.plot(history['val_loss'], label='validation')
pyplot.legend()
pyplot.show()
Test improved model on Validation Data¶
In [175]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
inv_yhat, inv_y, rmse=models.make_lstm_prediction(validation_X,validation_y,model,scaler)
print('LSTM Model on Validation Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload LSTM Model on Validation Data RMSE: 8.860
Test improved model on Unseen(Test) Data¶
In [176]:
# make a prediction
%load_ext autoreload
%autoreload 2
import models
inv_yhat, inv_y, rmse=models.make_lstm_prediction(test_X,test_y,model,scaler)
print('LSTM Moddel on Test Data RMSE: %.3f' % rmse)
The autoreload extension is already loaded. To reload it, use: %reload_ext autoreload LSTM Moddel on Test Data RMSE: 8.793